Ipswich took the lead through Jack Colback before Brett Ormerod equalised, paving the way for Owen Garvan to score the winner 13 minutes from time.
Charlie Adam had earlier missed a disputed penalty for Blackpool, with referee Eddie Ilderton the centre of attention for that decision as well as the sendings off of Ian Evatt and Rob Edwards.
Holloway stripped off his jacket and threw it to the ground in front of the dug-out when his second player was sent off and once in the stands, he shouted his disapproval at the refereeing during the match.
Holloway later said: "(Ipswich manager) Roy Keane wasn't happy with the penalty in the first half and I wasn't happy with the cards in the second. Why do I get sent to the stands for speaking to the officials in the same manner as Roy?
"I threw down my hat and coat because there was nothing else I could do. I let myself down but my players didn't let me down."
Keane said of Ilderton's decisions: "They were close ones. I am never one for wanting players to be sent off.
"I have got a lot of time for Ian Holloway. He is an emotional man. It is an emotional game. I did not hear him use any foul language towards the officials whatsoever."
Keane went on: "As far as our performance is concerned, we are turning in results now. We controlled the game well, although I was disappointed by the last 15 minutes. If you want to watch how not to play against nine men that was a display to watch."
It was the first meeting of the Championship clubs in the FA Cup and for Blackpool, it was their second successive third-round exit from the competition.