The England winger admits Martin O'Neill's side have to put on a better showing on Monday night than in their home league defeat to Wigan and the loss away to Rapid Vienna in the Europa League play-off.
But Milner takes heart from Villa's record of 10 league victories on the road during the 2008-09 campaign, including a win at Arsenal - although they were hammered 5-0 at Liverpool during a dismal end to the season.
Milner said: "I don't think it's a good or a bad thing that we've got Liverpool next. It doesn't really matter. You've got to play everyone at some point in the season.
"Obviously they had a bad result at the start of the season (at Tottenham) but they've come back strongly with a win against Stoke so we've got to bounce back quickly as well.
"It's never an easy place to go but it's early in the season and we need to find our form. We showed signs of that against Rapid, when we passed the ball around well, so we just need to get into our stride a bit quicker.
"It's a hard game whenever you play at Anfield but it's up to us to go there and play better than we have done in the two previous games and keep improving."
Milner added: "We obviously had a disappointing day at Liverpool last year.
"With the quality we've got here, we should never lose by that margin so it would be nice if we could bounce back and get all three points.
"We're a good team away from home as we proved at the Emirates Stadium last year so we have to go out and put on a performance.
"It's important and you always want to get your first points on the board as early as possible. We didn't do that on the first day so now we have the chance to go to Liverpool and get a good result."
Manager O'Neill points to how Villa suffered a similar early-season setback 12 months ago at Stoke but then responded in positive style to set up a challenge for a Champions League spot before their late fade-out.
O'Neill said: "You just don't forget these results but you get on with it. Look at Everton, they finished above us last season and had a great season but they had a bit of turmoil and got beaten (by Arsenal).
"We all have to fight back from things. It's a long season and I think it's about getting the first win.
"At a similar stage last season we went to Stoke and got beaten by a late goal and that was dispiriting.
"But we fought back against Liverpool, and drew at home with them, and then by winning away games at Tottenham and West Brom.
"We are capable of getting over it (the Wigan result)."
O'Neill added: "The disappointing aspect was really in terms of our overall performance. That can happen but it just wasn't good enough.
"I didn't expect that because our pre-season had been good in terms of playing lots of games quickly and trying to get fit.
"But it's early season. What you don't want is to get too far behind."
Villa centre-back Curtis Davies is doubtful for the trip to Anfield after a recurrence of his shoulder problem.
Davies suffered the setback during the latter stages of the Vienna clash and his replacement, rookie Shane Lowry, is amongst the candidates to replace him, although Habib Beye could switch from right-back.
O'Neill, who is continuing his pursuit of Sylvain Distin and Stephen Warnock to bolster his back four, will also be hoping striker John Carew is fit to make his first appearance of the season.
Carew, the club's top scorer for the past two seasons, missed the Wigan match with a foot injury and then pulled out of the Vienna game with a virus.
Liverpool v Aston Villa To Score in 90mins: Milner, J. 11/2