Gary Teale gave the Championship side an early lead before Paul Warne then levelled for the Millers shortly before half-time.
Kevin Ellison struck midway through the second half to fire the League Two hosts ahead but the Rams could not find a reply - sending them out of the competition at the first hurdle, in sharp contrast to last season's semi-final heroics.
Robins said: "We started the game brightly and had a couple of half chances. They scored with their first attack, but we didn't lose any heart from it.
"It was a big scalp for us and I am delighted. I am pleased with the manner of the performance and the manner of the win, and I am delighted for Paul Warne."
Source: Team_Talk
