Grant, normally known for keeping his feelings to himself, confronted referee Kevin Friend at half-time during Pompey's previous game and after victory against Alan Pardew's men headed onto the pitch again, this time to thank the away fans.
"I'm very emotional and this is an emotional game," Grant said. "If you win, you make 250,000 other people happy, this is what football is about.
Victory in the fifth-round tie means Pompey have bragging rights on the south coast and go into the quarter-final hat - but the bigger picture is that they have to return to the High Court to fight a winding-up order from Her Majesty's Revenue and Customs.
"I will fight until the last moment," Grant said. "As long as we have a chance on the pitch. What happens in court is out of my hands. If they need to punish someone then it is a person that does something wrong. I don't know who that is but it is not us."
The eventual scoreline flattered the visitors, who had David James to thank for keeping the scores level at half-time.
He pulled off fine saves to deny Papa Waigo N'Diaye and Wayne Thomas in the first half, then Adam Lallana just after the restart.
James is hoping the club will be saved, telling ITV: "My instinct says it will be. I think it's too important for too many people, for football even, that a top-flight club can go out of existence.
"It's not going to be easy and whoever it is who helps us out we'll thank them now, but it's important we stay in business and important Portsmouth keep flying the flag in the Premier League for Hampshire."
Midfielder O'Hara dedicated victory to the supporters who enjoyed the short trip along the south coast.
The on-loan Tottenham man said: "We had to give something back to the fans because it's been a bit of a dire season for us.
"The fans have been fantastic all season and it's good to give something back."
He added: "Hopefully we won't get Tottenham in the final because I wouldn't be able to play."
O'Hara was given the man-of-the-match award but the turning point of the tie came when Owusu-Abeyie, on loan from Spartak Moscow, was introduced just before the hour mark and scored within eight minutes.
"The only thing we could do is bring loan players in who were not under contract," Grant said. "Quincy was one of them. "I knew him from Arsenal and know that he has a lot of quality. I like to develop players and get the best from them and he will be very good if we get potential from him."
Grant was delighted with the result after such a traumatic week.
"If I wrote a book about Portsmouth it would be a best-seller," he said.
"I see the passion of everyone at the club - the fans, the players and the staff. There is a passion to succeed. I was very happy to see the decision from the High Court. There are many people around and it's not their fault.
"I had two choices. To give up or to show to the players that we will make our spirit even higher. I said that to the players and to my children when there are difficult times you need to show your character."
Saints boss Pardew was pleased with his side's performance until Pompey pulled away with their late goals.
"We tried to play a controlled game and not to open it up," he said.
"We did well with that. We were very disappointed not to be leading at half-time, David James was terrific, and it was evenly poised. Psychologically, scoring first would have been tough for Portsmouth.
"I think next time we'll play them we'll be in a stronger position. We are a lot closer than the team that played because we had players unavailable."