Simeon Jackson - who netted a hat-trick against Swindon on Saturday - and Andy Barcham struck within three minutes before half-time to send the Gills through.
Luke Summerfield pulled one back early in the second half but Sturrock continued his record as having never won a League Cup match as Argyle boss in five attempts.
He said: "I must have the worst record of all managers in cup competitions, it's unbelievable.
"One or two of my defenders didn't defend properly tonight. It's not proper to name people but there were some glaring attempts to clear the ball and that's where we had problems.
"We tried to change our whole philosophy of football. We concentrated on passing the football in pre-season and the one key factor for us is that we have forgotten how to defend.
"There were some basic errors out there and some of it was comical. For 40 minutes we were the better team but we go into the break 2-0 up and then after we pulled a goal back the game went long."
Visiting keeper Romain Larrieu was in the thick of the action early on denying Barcham and then pulling off a great save to keep out a poor clearance from his own defender David McNamee.
But the home side did make the breakthrough when Jackson capitalised on some good build-up play from Barcham and Curtis Weston.
Plymouth were still recovering when moments later Barcham headed home debutant Chris Palmer's corner-kick at the far post.
The visitors replied on 49 minutes courtesy of Summerfield's low free-kick from the edge of the area.
Gills sub Mark McCammon nearly sealed the victory with seven minutes but the chance of the night fell into injury time to Plymouth's Jamie Mackie.
The striker was clean through against Gillingham keeper Simon Royce but pulled his shot wide with the goal at his mercy.
And Gillingham manager Mark Stimson was thrilled with his team's display.
He said: "We weren't sure what sort of side Plymouth would put out but when we saw the team sheet we could tell they were taking it seriously and their manager went on record to say they wanted to progress.
"The move of the match led to our first goal and it was great play by Curtis Weston with a little one-two and the little man was in the right area to put it away.
"They should they gave absolutely everything. I keep going on about it but there are some quality players here and we have more of an opportunity to show that than last year because of the level we're at. We can surprise a few teams."