The Tractor Boys are out to close the gap on the play-off places, while the Saints are fighting for their Championship survival.
"We played better against Preston on Saturday than we did in our win at Queens Park Rangers the previous weekend," Gorman told GreenUn24.co.uk.
"Despite the 3-2 defeat at Deepdale our confidence levels are high and we have no fear going into the game against Southampton."
Gorman was chief scout at Southampton until last summer and still has a great affection for the club.
He added: "Like Ipswich, Southampton is a wonderful club with similar supporters and the chairmen are always looking to do what is best for the clubs."