Stiliyan Petrov's early own goal was cancelled out by Emile Heskey's equaliser, with James Milner establishing a lead for the visitors before England winger Downing struck late on.
Kanu pulled one back to set up an exciting finale but Ashley Young made sure of Villa's victory.
It means Avram Grant, who still needs paperwork to be completed before being officially confirmed as Pompey manager, can now concentrate on his club's survival campaign in the Premier League.
His first two matches in charge have resulted in defeat but a Carling Cup quarter-final was hardly the top of his priorities.
O'Neill's team selection, on the other hand, suggested he was deadly serious about the competition, with most of his first-choice players starting and Brad Guzan back in goal after his penalty heroics at Sunderland in the previous round.
Unfortunately for Grant, he was denied two of Pompey's best players this season as Kevin-Prince Boateng and Jamie O'Hara were ineligible.
Grant has been labelled an unlucky manager after John Terry's slip denied him the Champions League title - but after 10 minutes his Pompey side were ahead with a fortuitous opener.
Danny Webber's corner was poorly executed and should not have cleared Petrov at the near post, but the Villa skipper sliced into his own goal despite Milner being positioned on the line.
The lead only lasted two minutes though with Milner placing the ball beyond the Pompey defence and Heskey chipping over Portsmouth goalkeeper Asmir Begovic and in.
Heskey scored on his Villa debut in January at the same ground and it was his third club goal since that strike. In fact, Heskey had scored on his last three visits to Fratton Park as he had netted for Wigan before making the move to Villa nearly a year ago.
Encouraged by the equaliser, the visitors were ahead in the 27th minute through a classic strike from Milner. The England international cut inside from the left flank after collecting winger Young's throw-in and he gave Begovic no chance with his drive into the far corner.
Pompey's response was a shot from Hassan Yebda that went wide, Nadir Belhadj volleyed over, while Guzan had to dive at the feet of Webber after a poor clearance by Stephen Warnock.
However, Villa could have extended their lead before the break when winger Young crossed from the left - but Gabriel Agbonlahor's header went just over the crossbar.
Grant knew a one-goal lead at the interval could be retrieved - his Chelsea team ended up losing in the final of the competition two seasons ago after going in ahead at the break.
Villa, however, were proving difficult to break down, sitting deep to defend then attacking with pace, as shown through England winger Young who had an effort saved after exchanging passes with Heskey.
Tal Ben Haim also had to hack away when Heskey sent Warnock down the left channel and the full-back pulled back to the danger area.
It needed something special to get Pompey back in the game but Anthony Vanden Borre's blatant simulation in a bid to win a penalty was anything but, and referee Lee Mason booked the Belgian for his pathetic dive.
Downing's moment came 16 minutes from full-time. Winger Young sent over a cross from the right that took a wicked deflection and span across Begovic's goal - and Downing judged the bounce perfectly to head home.
Kanu bundled in a scrappy goal from close range, but Young immediately tucked away Agbonlahor's cross to restore the two-goal cushion.