The Ibrox men drew a blank at Kilmarnock on Saturday but in the dying stages of an exciting game at Parkhead, with Chris Killen's header having levelled Suso Santana's early opener and substitute Niall McGinn denied a penalty claim when he appeared to be chopped down by Lee Wallace, it looked like Mowbray's side would come up short again.
However, defender Glenn Loovens dramatically headed in the winner three minutes into added time to take the Hoops two points clear.
Mowbray, who, in his excitement, swore in a radio interview immediately after the game, said: "I think there was a little bit of deja vu from the Dundee United game last week.
"We concede a goal early in the game, we go chasing and we had another controversial penalty decision at the death that didn't go our way.
"But thankfully we didn't need it and we got the winner.
"I thought it was a penalty and I was 60 yards away and the referee was 10 yards away, so let's hope the referee got it right.
"It doesn't matter, we didn't get the penalty as we didn't last week but I'm not too concerned."
Mowbray played down the anxiety which cascaded from the stands until the home side grabbed the winner.
"The fans react to the scoreline," he said.
"There are expectations at this club, but I am a coach who judges by performance levels.
"If your performance level is good enough each week then you will win enough football matches.
"But I understand that at this club, while it is not unique within the Scottish game, we have to win regardless of how we might play.
"Celtic fans don't want me to sit there and say 'I thought we played great' and we have lost 1-0.
"They want to win and hopefully they went home happy and are talking about some of the good football that was played today. I thought the team played exceptionally well."
Celtic defender Andreas Hinkel shrugged off the anxiety among the Hoops support, claiming that he had been through worse when he played for Stuttgart earlier in his career.
"You feel it a little but to be honest it is not a problem for me," the Germany international said.
"In Stuttgart the fans are very critical, it is different here, the fans are very good.
"I had it as a young player at Stuttgart and I was struggling because they were whistling and things like that but I have more experience.
"Maybe some of my team-mates are affected, it is a new situation for some of them but if they play longer they will get used to it."
Hearts manager Csaba Laszlo was disappointed by costly errors of judgement by two of his players but was more puzzled at how referee Dougie McDonald had arrived at four minutes of injury time.
He said: "I am disappointed to lose in the last seconds and we had two bad decisions from my players, Lee Wallace and Suso Santana.
"Lee thought he was 100% sure that it was a goal kick so he let it go but it was a corner and they scored the winner from it.
"For the equaliser, Santana had the chance to clear but he thought it was a foul.
"I accept their decisions, it can happen with anybody, maybe next time they will be more clever or make a different decision.
"The biggest question I had was where four minutes of injury time came from?
"I can't give you the answer, it wasn't me who gave four minutes, you must ask the referee."