Struggling Saints remain rooted to the foot of Coca-Cola League One after being held to a 0-0 draw by 10-men at St. Mary's.
Colchester were reduced in number when Kem Izzet lunged in and sent Saints youngster Adam Lallana flying with a scything challenge in the 35th minute, earning himself a red card.
Both sides had chances to win the game but neither found the back of the net.
"I'm disappointed for the guys because we were good enough to win," said Pardew. "I thought Colchester defended superbly for long, long periods of the game, but chances still came our way thick and fast and we probably deserved the win.
"The team lacks a real athletic threat and a bit of pace but I can feel a little bit of nervousness about us too. We could have capitalised on them but we were too edgy."