Gillingham were left seething at the perceived injustice of the decisive penalty, but Martin O'Neill was relieved to lead his Villa players away triumphant from a battle he described as 'blood and thunder'.
Milner, on his 23rd birthday, provided Villa with their reprieve after Simeon Jackson equalised for the League Two side with a polished finish.
Rather than contemplating a glamorous replay at Villa Park, the more prosaic reality that Jackson faced was a league match against Aldershot next weekend for his next assignment. And all that Gillingham captain Adam Miller could cling to was the shirt given to him by Villa's Ashey Young – ironic, given that the winger had, just minutes earlier, found himself the focus of his opponents' ire.
It was Miller who had sent Young sprawling theatrically in the box to concede the penalty and, starstruck though he was by the sight of the shirt, he was not about to shelve his grievances. 'Everybody saw that it was soft and believe me, it was soft,' he said.
Young was barracked from all sides on this numbingly cold Kent afternoon and Mark Stimson, the Gillingham manager, was inclined to sympathise with the chastened home supporters. 'I think it was harsh but it's a lesson to learn,' he said. 'But if you're going to go for the ball, you've got to be 100 per cent sure, otherwise you give the decision over to the referee and sometimes they do get them wrong.'
O'Neill professed to have had no view of the same incident and spoke only of his satisfaction that Villa had surmounted a 'very difficult' test. Gillingham had torn into their more exalted opponents with abandon, Miller reflecting ruefully that their impressive, if brief, exposure on lunchtime television was 'over too quickly'.
O'Neill said: 'If the players didn't know beforehand they were in some game, after about 10 minutes they did. We got a great goal and it settled us down but I never felt comfortable. It [the outcome] was in the balance.'
The balance was tipped by Milner's acute eye for goal; the midfielder having stepped admirably into the breach left by Gabriel Agbonlahor, who had called in sick.
The most innocuous errors cost Gillingham, as was illustrated early on when Miller lost possession on the edge of the area only to find Milner, combining clinically with Stiliyan Petrov, ready to pounce with an elegant strike.
But Villa were not uniformly convincing, the teenager Nathan Delfouneso unsteady in attack and Curtis Davies culpable for a clutch of botched clearances.
Such flaws fed a gnawing uncertainty, underlined when Jackson angled an unstoppable shot beyond Brad Friedel. But, with 11 minutes left, Miller recklessly dived in, Young went down, Milner did the rest, and natural order was restored.