That would allow the players an easier and longer recovery ahead of the Premier League's 18th-placed club's important trip to play Hull on Sunday.
"From our point of view, it is a case of getting through to the next round in the 90 minutes as quick and as clinical as we possibly can," said Allardyce.
"We want to try to avoid it from going to extra time, but if that is the way it has to be, and we have to do it the hard way again, then we'll do it.
"There are a lot of games coming up in this period so the momentum of getting back to winning ways can be taken from the cup and into the league.
"Let's not waste it."
Midfielders Vince Grella and David Dunn are struggling with fitness but Allardyce is certain to make changes in order to keep his players fresh for their important battle against Barclays Premier League relegation.
"We have to use the full potential of our squad, which is pretty fit at the moment and needs to be," he added.
"We have to make sure they deliver the sort of performances they have delivered to get us this far."
Injury and illness have meant left-back Martin Olsson has played just one match - the FA Cup third-round victory at Blyth - for Allardyce since he took over from the sacked Paul Ince but he is likely to feature tonight.
"I've been out with injury and viruses but if I get my chance I'll do my best," he said.
"We have to be focused and start playing how we do and I think we will be fine.
"We are playing good football when we are playing together. We know we are good as a team and we know we can beat Coventry even if we don't have the strongest squad.
"If he (Allardyce) plays the young ones or the ones who don't play every week then the others can think about the Premier League games.
"It is an important game because we want to go to Wembley but the Premier League is more important and we have to focus on that."