Sunday's News of the World claims Milner would not countenance a switch to Old Trafford due to his Leeds United connections. The 24-year-old was born and raised in the city and made 48 appearances for the Whites before moving to Newcastle for £3.6million in 2004.
His value has risen significantly in the last six years, however, so much so that Villa boss Martin O'Neill is now said to be looking for around £40million if Milner were to be sold.
That would mean Manchester City, who have already made an initial approach of £20million, would have to significantly raise their bid in order to land Milner.
Chelsea are also said to be interested, while the tabloid adds that Milner's first preference would be Liverpool - but only if he could follow O'Neill to Anfield.
The Villa boss has been linked with the Reds job should Rafa Benitez move on this summer.
Meanwhile, Sunday's papers continue to link Tottenham's Republic of Ireland striker Robbie Keane with a switch to Villa Park. The 29-year-old, who spent the second half of the season on loan at Celtic, would probably cost around £15million.
Also, giant Serbia striker Nikola Zigic, 29, and Portugal right wing-back Miguel, 30, have also been suggested as Villa targets. The Valencia duo would cost a combined fee of around £9.5million.