Emile Heskey had put the visitors in front with 24 minutes gone and, although the Black Cats more than made a game of it, Stewart Downing and Ashley Young both hit the woodwork before Milner made sure of the points.
Steve Bruce's men might have got themselves back into the game four minutes after the break but Andy Reid wasted arguably their best chance of the game, and Villa were in no mood to allow their advantage to slip as they became only the second away team after Chelsea to win at the Stadium of Light in the league this season.
Sunderland, who introduced Lee Cattermole as a late substitute on his return from a knee injury and then saw skipper Lorik Cana sent off for a second bookable offence, have now won only one of their last eight top-flight fixtures, and the bulk of a crowd of 34,821 left reflecting upon a rapidly-dissolving good start to the campaign.
If ever they needed a reminder that football can be a game of fine margins, Sunderland were given it in a frustrating 90 minutes.
Having surrendered a 1-0 lead with literally seconds of injury time to play against struggling Portsmouth at the weekend, they found themselves trailing at the end of a first half during which they had gone toe to toe with one of the Premier League's form teams.
Jordan Henderson might have set the visitors back on their heels within eight minutes of kick-off when he rifled a blistering long-range drive inches wide of the angle of bar and post.
By the same token, Villa went even closer with 16 minutes gone when, after Heskey had been felled by George McCartney, Downing, who had a loan spell at the Stadium of Light earlier in his career, struck the crossbar with a free-kick that had keeper Marton Fulop beaten.
The breakthrough finally arrived eight minutes later, and again McCartney was the culprit.
His aimless cross-field ball was picked off by the ever-alert Stiliyan Petrov and, when he fed Milner, the England midfielder slid the perfect pass into Heskey's path for the striker to smash home.
Sunderland's response, however, was spirited and, after Kenwyne Jones had stabbed a 35th-minute shot wide, they were very nearly back on terms.
The impressive Henderson found space on the right to float a cross to the far post, where Reid volleyed right-footed towards goal, only for keeper Brad Friedel to save with his legs.
Kieran Richardson sliced wide from 20 yards after Richard Dunne had only just beaten Jones to Reid's cross and Henderson fired high over the bar with a minute of the half remaining, with the home side pressing hard.
There were boos from the locals as the whistle sounded, although they were reserved exclusively for referee Kevin Friend and his assistants amid a red and white perception that their side had been on the wrong end of too many tight decisions.
But they were on their feet within four minutes of the restart as the path to goal opened up invitingly for Reid.
Bent climbed well to flick McCartney's long ball perfectly into the midfielder's pass and he looked certain to score but sliced his left-foot shot harmlessly wide.
The game was becoming increasingly open, with Heskey and Paulo da Silva firing in efforts at either end in quick succession.
However, Sunderland needed the help of the woodwork for a second time with 57 minutes gone when Ashley Young made the most of the space he was afforded on the edge of the box to run on to Heskey's pass and shoot against the post under pressure from Michael Turner.
But no-one could help Fulop four minutes later when Milner picked up the ball 30 yards out and, having steadied himself, launched a missile into the top corner with the Hungarian powerless.
Dunne had to be at his best to prevent Jones from converting McCartney's low cross five minutes later but Sunderland's dwindling hopes effectively died with 12 minutes remaining when Cana, who had been booked for a first-half foul on Gabriel Agbonlahor, picked up a second caution for a late challenge on Milner and was dismissed.
Substitute Phil Bardsley flashed a late header just wide but there was to be no consolation for the home side.