Millers forward Paul Warne had a shot saved in the sixth minute and then Tom Pope went close with a diving header.
At the other end Will Buckley was denied by a decent stop from Andy Warrington, and the Rotherham keeper was called into action again five minutes later to keep out a Chris Dagnall shot.
Le Fondre scored his second goal in as many starts for the Millers in the 43rd minute, managing to get his head to Nicky Law's free-kick in a crowded goalmouth.
But Rochdale hit back almost immediately. In the second minute of added time Dagnall latched on to Gary Jones' through ball, and although his first shot was blocked by Warrington, the striker followed up to squeeze the ball home.
It was veteran forward Paul Warne who stole the points for the home side with his third goal of the season on the hour. Pope crossed from the right and Warne's delicate touch at the near post turned the ball into the net. Rotherham were forced to defend late one, with Warrington saving well from Chris O'Grady in injury time.