The visitors were all over the place as Killie raced in to an astonishing interval lead through goals by Dean Shiels, Paul Heffernan and James Fowler.
The Parkhead boss had to work hard at the break to avoid an embarrassing defeat and a severe blow to his Scottish Premier League title hopes and his players responded with three goals in six minutes to take a point.
Anthony Stokes scored twice, the first from a free-kick in the 73rd minute, before stand-in skipper Charlie Mulgrew headed an equaliser from close range in the 79th minute.
Afterwards Lennon revealed he was well aware the pressure he was under as he made his way up the tunnel.
He said: "I was really angry as I walked off and I thought to myself, 'this is probably your biggest test yet, go and see what you can do and see if you get a response from your players'.
"Another defeat and the manner in which we were playing would have been pretty difficult for me to take.
"So I was angry as I could possibly be. The defence was very poor, the midfield was poor and the two front boys were poor so as a collective performance it was nowhere near acceptable.
"Thankfully we did respond but it was nothing to do with tactics or anything like that it is about motivating the players and pushing them to dig deep to get something out of the game.
"I asked them to find the belief, the will, the guts to dig deep and get something out of the game and they did that.
"But there were mixed emotions; anger and also I was very proud of their second-half performance.
"Overall I'm disappointed that we dropped two points.
"I think I will get it right, I just want to make sure it's not too late."
Stokes admitted the Celtic players knew at the interval that the pressure was heaped on Lennon.
"The lads stood up for him in the second half," he said.
"There would have been a lot of pressure on the gaffer so we did it for him, ourselves and the fans because they deserved it as well as the first half was unacceptable.
"It was a catalogue of errors.
"I missed a sitter in the first half but I repaid the gaffer in the second half.
"We knew at half-time it wasn't good enough and we had to improve massively and we showed a lot of spirit to get back into the game."
Kenny Shiels would not blame Heffernan for the striker's last-gasp miss which would have given the home side three points.
The Killie boss said: "He knows he should have scored and a player of his quality and natural goalscoring instincts, as he showed in the first half, normally puts them away because even though he is small he is good in the air.
"So I would have expected him to score and he would as well but we don't hammer him."
The Northern Irishman was pleased by the way his side continued to play even when their lead was being chipped away.
"When you are 3-0 up after 72 minutes you are looking to get the three points," he said.
"But when it is against the Old Firm there is always quality in there which means they can get back into the game.
"But it wasn't as if Celtic's goals came from incessant pressure.
"It wasn't like that, we had two good chances at 3-3 which was testament to the players' character that they didn't sit back and look for a point."
Heffernan said: "I am disappointed, obviously, it was a great chance to win the game in the last couple of minutes.
"I got too much on it and put it over the bar.
"We were three nil up after 70 minutes so we are all disappointed.
"It feels like we have lost - but I'm sure they feel the same."