The match turned out to be one of their most memorable matches in history.
Under-fire manager Martin Jol witnessed his Spurs side fall 4-1 behind despite taking a 20th minute lead through Dimitar Berbatov.
A brace from Danish defender Martin Laursen gave Villa the lead and goals from Gabriel Agbonlahor and Craig Gardner soon made it 4-1.
Pascal Chimbonda scored the rebound after Jermain Defoe's effort hit the post.
A clumsy foul from Marlon Harewood on Darren Bent gave Spurs a lifeline from the penalty spot, with Robbie Keane dispatching the spot-kick.
It looked like Villa would hold on, before Kaboul smashed in a stoppage-time winner.