Naylor saw his free-kick deflected in early in the first half before Kenny Miller equalised.
Marc Antoine Fortune restored the advantage on the stroke of half-time with what proved to be the winner before Lee McCulloch saw red in the last minute.
Lennon has been in charge since Tony Mowbray left the club and Naylor hopes he gets the chance to lead the club next season when they bid to win the Scottish Premier League title back off Rangers.
Naylor said of Lennon on Sky Sports 3: "He's put me straight back in and he knows what I can do. I hope he gets the job because he's done a terrific job."
Naylor is on a free transfer in the summer but hopes Lennon will want to retain his services.
He added: "I'm on a free in the summer so we will see what happens. I've got a lot of thinking to do over the summer so we'll see."
Rangers manager Walter Smith admitted his side paid the price for a slow start despite creating a lot of chances in the second half.
He told Sky Sports 3: "I felt Celtic started very brightly. We never really got going in the first half.
"In the second half I felt we played far better. We kept the ball and created more opportunities.
"We had chances there, we hit the post and the goalkeeper made a couple of good saves. We have got to accept it, we had the chances."
Lennon admitted he wanted to job on a permanent basis.
He said: "I don't know, it's up to the powers to be. I feel I've done enough."
Neil Lennon praised his players for their commitment.
"I thought the team were magnificent. Overall they were magnificent tonight. It's not fair to single people out, it was a great team effort."
Lennon was also pleased for Naylor and said: "He's had his critics here. He's a quality player and he's been terrific for me since he's come back into the team."