Houllier watched Villa's 1-1 draw with Bolton from the directors' box alongside new assistant Gary McAllister.
The Frenchman's enviable managerial record includes winning league titles with Paris Saint-Germain and twice with Lyon, as well as the UEFA Cup, FA Cup and League Cup - twice - with Liverpool.
He takes over from MacDonald, who has spent the last six weeks in caretaker charge following the departure of Martin O'Neill.
Agbonlahor said: "If you look at his record at Liverpool, what he has done, what he has won, obviously we're looking forward.
"It's hard to say anything because Kev (MacDonald) has done so well, but times change and we'll be looking forward to training with the new manager."
It is clear there is an affinity towards MacDonald, highlighted by Ashley Young's celebrations after opening the scoring against Bolton with a superb 24-yard free-kick in the 13th minute.
After seeing the ball ripple the net, Young turned, pointed towards an applauding MacDonald before racing over to the Scot standing on the touchline and giving him an embrace, with several other players joining in.
Asked whether there was any sadness MacDonald had not been offered the role on a permanent basis, Agbonlahor added: "It's hard to say that.
"We don't want to discuss that now because there is a new manager in charge, so we have to look forward to that and be happy with what Kev did for us.
"He'll still be around so we'll still be able to see him and thank him.
"But I think you could see by the goal, how everyone celebrated, that it shows how well we've enjoyed working with him. He's been a good manager."
The appointment of Houllier has given Villa the chance to make long-term plans.
"It has been frustrating not knowing what was going on, but at least now we know and we can look forward to the future," added 23-year-old Agbonlahor.
"We're just going to get our heads down as a team because we want to try and impress the new manager.
"We don't know what he knows about us, so hopefully it will be a fresh start for everyone in the squad.
"We'll see what ideas he has and what training will be like, and we'll take it from there."
After conceding 13 goals in their three previous visits to Villa Park, Bolton arguably caught a Villa team in transition at the right time.
Captain Kevin Davies' 35th-minute equaliser was well deserved, and but for three fine saves from Brad Friedel in the second half then it could have been so much more.
The Houllier effect certainly worked for Bolton as Davies said: "It looked like their players were a little unsure as to what was going on.
"Sometimes as a player if you are unsure, you've someone standing in, then it can affect you, and I thought at times it looked like that."
Davies, who shook off the effects of concussion sustained in last weekend's 4-1 defeat at Arsenal, was dismayed not to have come away with a win that would have lifted Bolton to fifth.
"I've mixed emotions because I'm pleased with a point because we don't normally get anything here, but also disappointed we've not won," added Davies.
"It was a great performance from us, absolutely outstanding.
"Maybe we're lacking a bit of belief sometimes, but you could tell by the reaction of their fans they were disappointed, and that they were probably happy to escape with a point."