Dunn's winner capped a stirring comeback from Rovers, who had fallen behind to Gabriel Agbonlahor's third-minute strike then saw Vince Grella sent off for two bookable offences.
Allardyce said: "I would have been happy with a draw but I'm glad to say my players weren't. They started attacking and we got what we deserved."
He added: "I'm delighted at the way we kicked on after going an early goal behind from such a sloppy piece of defending. But it only knocked us back for a bit. We came storming back and Morten Gamst Pedersen had a good chance and we got our well-deserved equaliser.
"You rarely come back from behind to beat a top four or five team and we have done it in the last half hour and with 10 men.
"It's a day I'll remember for a long time. It's a great day. Days like this are what I do this job for."
Allardyce singled out Dunn as the driving force behind his side's second-half revival, adding: "Dunny was outstanding. I think he would have scored in the first half if Brad Friedel hadn't pulled off a world-class save."
Allardyce added that he had no complaints over Grella's red card and even had rare words of praise for referee Mark Clattenburg, adding: "I've got no complaints, they were clearly two bookable offences, the referee got the big decisions right."
Villa boss Martin O'Neill was disappointed at his side's failure to build on Agbonlahor's goal and ultimately see their 100% away league record go up in smoke.
O'Neill said: "I'm naturally disappointed to lose all three points. We got off to a great start and I thought even at 1-1 at half-time we would go on to win. Maybe we didn't create as many clear-cut chances in the second half but I thought we were worthy of a point but it's never easy here."