Wednesday's third-round encounter will be former Liverpool boss Houllier's first match in charge of Villa since his appointment as manager earlier this month.
Allardyce is well aware of Villa's potency in the cups having seen his side eliminated by them in both the FA Cup and Carling Cup last season - in the latter case, at the semi-final stage - and the Rovers boss is expecting another tricky task now the opposition have a new man at the helm to impress.
"It will be good to see Gerard again," Allardyce said.
"He has been involved in many ventures since he left Liverpool - he has been in close touch with football throughout Europe, he has had his position with the French FA and also his job at Lyon.
"He is obviously very eager to get back into this country because he knows it is the best league in the world.
"To coach, manage or play at this level is what everybody wants to do throughout the world of football.
"He will be delighted to be back and it will be a difficult game for us now.
"It was always going to be a difficult game, but I think the players that he will pick will be going all out to impress him."
Allardyce believes Houllier will field a strong starting line-up as he looks to make a positive start in his new job.
"I think that Gerard will be picking the strongest side because he will want to get off to a winning start at Villa Park," he said.
"I managed a 3-0 win against Stoke City in my first game [as Blackburn manager].
"That was a major relief for everybody and it gets everyone's confidence up and settles everyone down."