The Gills gave a good account of themselves against a strong Rovers side, but goals from David Dunn, David Hoilett and Morten Gamst Pedersen saw the Premier League outfit earn a place in the third round.
Dunn's goal came in just the fifth minute after Rovers flew out of the blocks but the Gills came back into the game and could have gone in level at the break.
Hoilett doubled the lead in the first minute of the second half and although Simeon Jackson got one back from the spot, Pedersen's brilliant corner went in via the unlucky Barry Fuller to seal the win.
"They gave us a bit of a run for our money in the first half," said Allardyce. "We got a nice goal which I thought would set the tone for us but we sort of slipped off the pace a little bit and let them get back into the game.
"We just had to sort that out at half-time.
"We had to just stop giving the ball away as much as we did, but that was down to some of the hard work that Gillingham put in to it, but when we went out in the second half and kept the ball and got the second goal it knocked the stuffing out of them."
The penalty was given for a foul by Zurab Khizanishvili on Jackson and although his players were unhappy, Allardyce's view of the incident was not clear - and it mattered not after Pedersen's goal.
"We missed another three or four chances that would have made it comfortable," he added. "Then we got a little edgy for a while and then the penalty was given.
"It was a little too far away for me to know whether it was or it wasn't, the lads were complaining about it but we got the goal, a great goal from Morten.
"I thought he deserved the goal. I think it was going in anyway but went in off the defender."
Allardyce saw Steven Reid go off at the break with a hamstring injury and also took off full debutant Nikola Kalinic at the interval.
"The only disappointment was that Reidy got a little hamstring injury, nice to see him back and Dunny looked really good with his goal and general ability.
"Nik felt a groin at half-time so we took him off but you can see the movement and where he wants to go but he's just not quite up to the pace of our game yet so that'll come I think.
"All in all a good night and we travel home in the hat and look forward to the next round."
Gillingham manager Mark Stimson was pleased with his side's performance after their weekend horror show against Hartlepool and hailed the performance of Pedersen.
He said: "I won't go home happy because we've lost, but the performance tonight to Saturday was a million miles better because we had a go. We had a go and unfortunately at the end the quality showed.
"At 2-1 you've got a chance and then the kid puts a corner in and it was unstoppable really. The lad (Jason) Roberts has done well on Simon Royce, he's just blocked him but the ball in was absolutely fantastic.
"If I'm honest, (Pedersen) was the one player we would have loved to be left out tonight, because he can unlock the door.
"When we saw his name on the sheet we thought 'why him?' and he's undone us in the end.
"I hope the fans can see what we're trying to build - we're trying to build a team that attack and are creative."