Anthony Elding hit the post late on for Stockport but in the end a draw was probably just about right after a thrilling game.
Both sides made just the one change to their starting line-ups for the clash at Edgeley Park.
Stockport drafted in two-goal striker Liam Dickinson for his first start of the season, with Elding moving out to the left, while Rotherham replaced midfielder Danny Harrison with Steve Brogan.
Striker Matty McNeil twice fluffed near-post efforts for Stockport, while at the other end goalkeeper Conrad Logan produced a smart save from Chris O'Grady.
A blistering start saw chances galore at both ends, before Stockport eventually went into a deserved 13th minute lead when David Poole's excellent inswinging cross was headed home by the unmarked Elding, for his first goal of the season.
There was then a relative lull in the game, before Rotherham came mightily close on 38 minutes when Dale Tonge's looping cross was headed just wide by Derek Holmes.
Minutes after the restart Holmes was again unlucky, this time after Logan superbly beat away his fierce drive from fully 20 yards.
The referee was the centre of attention as the visitors levelled on 55 minutes when he whistled just as Brogan took a free-kick. The Stockport defence stopped in the confusion and Ian Sharps rose unchallenged to make it 1-1.
County boss Jim Gannon was subsequently sent to the stands following his protests.
A dramatic spell then produced two goals in two minutes. Substitute Adam Griffin headed Stockport in front direct from Adam Proudlock's corner, then McNeil's own goal levelled it up again as he headed home Brogan's inswinging free kick.