Keane's side took the lead at London Road thanks to Tamas Priskin's first goal for the club since his £1.7million move from Watford.
But Priskin then saw a penalty saved by Peterborough goalkeeper Joe Lewis before Posh fought back through goals from Lee Frecklington and George Boyd.
Boyd's winner came in the 64th minute when Toumani Diagouraga won the ball 30 yards from goal with a crunching tackle and the ball span out to the left wing.
Boyd skipped around a flimsy challenge by David Wright and crashed a low shot through Richard Wright at his near post.
Keane's former Manchester United team-mate Darren Ferguson will now lead Posh into the third round while Keane is still looking for his first win of the season.
Keane said: "The penalty miss was a big factor in the game of course but if you don't take the opportunity you still have to deal with that.
"If we had gone 2-0 up it would have been very difficult for Peterborough to get back in the game.
"We started well and when we got the ball down and passed it we looked a decent team but the second goal was a major disappointment.
"The first goal was an outstanding strike but the second was very poor defending. I've said it before but you just can't give away goals like that."
Keane refused to comment on specific transfer targets but admitted that he is still hopeful of adding one or two players to his squad before next Tuesday's transfer deadline.
He said: "There is no progress on the transfer front and there are no guarantees.
"We will have to see what the next few days bring us. I don't want to talk about any players who may come here as we may have to deal with who is still here."
Sir Alex Ferguson was joined in the stands by Derby manager Nigel Clough and Thailand boss Peter Reid amongst others but Peterborough boss Darren Ferguson is confident that the likes of Boyd will remain at London Road.
He said: "There are a lot of managers interested in my players but last season we had 15 or 16 teams watching us some weeks so we are used to it.
"We will not sell any of our players unless it's a ridiculous bid so I am not concerned. I am backed by the chairman here and there is no chance of any of our best players leaving this month.
"We needed to stop the rot but it doesn't mean anything unless we get a win next Monday night against Crystal Palace."