The home side struggled to come to terms with their opponents passing game early on and trailed to Dean Shiels' opener before Andy Carroll and Kevin Nolan - in the 92nd minute - sent Newcastle back to the top of the table.
Zurab Khizanishvili was also sent off for United, who could have gone on to lose the game had Doncaster's Martin Woods not missed a 76th-minute penalty.
"I'm more relieved because at 1-1 they had an opportunity to go 2-1 up," Hughton said.
"I'm certainly relieved that it (the penalty) didn't go in. I thought on the balance of the game that we deserved to win it.
In the first half they (Doncaster) played a good tactical game but they will keep possession against any team.
"I thought over the 90 minutes that if there was a side that was going to win the game I felt it was us but at 1-1 and with the penalty you think the worst."
Hughton is set to meet with owner Mike Ashley this weekend to thrash out a deal to make him the club's permanent manager and, although he had no update on the situation, he still expects to be installed as the new boss without a hitch.
He also reserved praise for the spirit of his players, who ended the game to a standing ovation after often being heavily heckled by a season-high crowd of 43,949.
"I think it says everything (about our spirit)," he said. "We're realistic enough to know we've had a difficult time in recent weeks.
"Our performances in the last two games were very good and the two performances here (at St James' Park) before that we found difficult.
"There is a lot of character in that side. To get the winner when we did speaks volumes for this group of lads."
Doncaster boss Sean O'Driscoll refused to dwell on Woods' penalty miss, insisting he would have been disappointed with aspects of their game had they won or lost.
After leading early on, Rovers could have regained their advantage after 75 minutes when Ryan Taylor handled in the box, only for Woods to fire six feet wide of the goal.
"If my auntie was my uncle," was O'Driscoll's phlegmatic verdict of the miss. "I would have said the same if we'd have come off winning 1-0.
"There's no point hiding behind that, we pride ourselves on doing certain things and at times I thought we didn't do them very well.
"He (Woods) is the skipper. Penalties will be scored and missed as long as there are football matches. He's taken responsibility and the modern game needs players to take responsibility.
"All we ask is that if they make a mistake they react in a positive manner so in training on Monday morning he's got to be bright."
Despite passing Newcastle off the pitch for the best part of an hour, O'Driscoll was critical of his side's ball retention and claimed he would have felt the same way had they have won the game.
"I would have said the same had we won 1-0, drawn 1-1 or lost 2-1; we need to be better with the ball," he said.
"For us that's the big aim. When you come off a game you need to learn from it and just because we've lost I can't throw my toys out of the pram and moan about things I don't think are relevant.
"We're disappointed but if we want to turn a 2-1 defeat into a 2-1 win then we have to be better with the ball."