Villa substitute Nathan Delfouneso scored with his first touch in the 82nd minute to give the visitors victory at Fratton Park.
David James saved John Carew's penalty before the break after the Norwegian had restored parity following Michael Brown's early opener.
Delfouneso tapped home Emile Heskey's flick-on to move Villa above Liverpool into sixth place in the table, and England midfielder Milner insists the six-point gap to current fourth-placed side Tottenham is not insurmountable with four games remaining.
"We've said all the way through to just keep playing our football and we're still in the hunt," Milner told Sky Sports.
"It's going to be tough, we're going to need a bit of help but we just need to concentrate on our games and get as many points as we can.
"If we can get maximum points from the final games then we've done our bit."
Asked if fourth place was realistic, he added: "I don't see why not. We're still in there but we're going to need a bit of help to finish where we want to, and that's obviously fourth place.
"There's always ups and downs, and teams put runs together."
Milner admitted Portsmouth, already relegated but in the final of the FA Cup and playing now for pride and places at Wembley, were difficult opponents.
"They're obviously on a high, they've just got to the final," he said. "It's a tough place to come and they're playing with freedom now that they're down.
"They're going out there and enjoying it. It makes it tough."
He added: "The pitch was quite heavy and it was very warm, so it was tough for both teams.
"We created a lot of chances, moved the ball well, and probably could have scored more goals."
Villa boss O'Neill heaped praise on matchwinner Delfouneso, telling Sky Sports: "I'm delighted for him.
"I've been saying this for some considerable time, but he's going to be a proper player and if he keeps his feet on the ground, he's got a real proper chance.
"He's got real ability and, up to date, his attitude has been excellent."
O'Neill echoed Milner's view that Villa can only look after their own results - with their next outing on Wednesday at Hull - and hope that will be enough to reach the Champions League come the end of the season.
"To give ourselves any chance we have to win all our games, and today would have been disappointing, considering the amount of possession we had away from home, not to have taken all three points," he said.
"I'm delighted we've done that and we go to Hull on Wednesday and we're still full of hope."