City are believed to have tabled a firm offer for the England utility man, while Chelsea are also thought to be keeping a watching brief on him.
Milner has established himself as one of the most sought-after players in England after a superb campaign for Villa.
It has been suggested Villa may have to sell before they can buy this summer, with Milner one of their most saleable assets.
However, Lerner insists the former Leeds and Newcastle star - who can play in several positions in both midfield and defence - will remain at Villa Park next season.
"I'm not worried about losing James Milner. I think James will play for Aston Villa and should play for Aston Villa," Lerner told the Daily Mail.
"I'm not dismissive of the realities that a player that has received the attention and acclaim of James is going to spark discussion and debate about a move and so on. The answer is "No", I don't worry about him leaving because he is wanted at this club."
Milner, who is part of England's World Cup squad, was named PFA Young Player of the Year last season and he is set to find himself involved in a transfer battle this summer.