The £3.8million signing from Nancy, who has failed to impress since arriving at Parkhead, replaced Danny Fox and, with almost his first touch, fired a glorious shot past Motherwell goalkeeper John Ruddy.
Georgios Samaras' 14th-minute opener had been cancelled out before the interval by goals from Lukas Jutkiewicz and Mark Reynolds.
Aiden McGeady levelled in the 51st minute and Barry Robson had a penalty saved by Ruddy before Fortune fired in the winner.
In freezing fog at Fir Park, 18-year-old debutant Steven Meechan was joined by Scotland Under-19 team-mates Robert McHugh and Steven Saunders in a Motherwell side with an average age of just over 21.
And in the first 20 minutes they looked as if they would be blown away as the visitors started in impressive fashion.
In the eighth minute, the visitors came close when Samaras found space inside the area but his angled shot struck the legs of Ruddy before rebounding off the crossbar and away to safety.
However, the Greece international did better in the 14th minute when he took a pass from Landry N'Guemo at the edge of the box, shrugged off the attentions of a couple of Motherwell defenders and rifled his right-footed drive from 16 yards high past Ruddy.
Ruddy then had to push McGeady's 25-yard drive out for a corner before making another save from the same player's low drive moments later.
In the 25th minute, with the Hoops defence in disarray for the first time, Hinkel had to put in a saving tackle on Jutkiewicz, who was shaping to shoot just 10 yards from goal.
However, a minute later the Well striker made up for his hesitancy when he levelled with a clever near-post flick from McHugh's cross, although Artur Boruc did not cover himself in glory with his positioning.
Celtic were momentarily stunned but skipper Gary Caldwell headed a Barry Robson corner just past the post as they
tried to reassert their dominance.
Five minutes from the interval, as Celtic broke, Reynolds tripped Samaras inside the box but referee Charlie Richmond ignored the penalty claims.
That decision became more pertinent in the final seconds of the first half when, from Tom Hateley's free-kick which was inadvertently helped on by Glenn Loovens, Reynolds scored from close range to dramatically put Motherwell ahead.
Celtic started the second half determined to get back level and there was some nervous and erratic play from the home side as they tried to withstand the initial pressure.
But in the 51st minute, after Motherwell had again failed to clear their lines, Celtic levelled.
Samaras slipped in McGeady just inside the box and the Republic of Ireland star turned back onto his left foot before curling the ball high past Ruddy.
Moments later, Samaras blasted over the bar from 20 yards after being set up by Scott McDonald.
In the 55th minute, as Motherwell defended desperately, the ball appeared to strike the arm of Steve Jennings inside his own box but again Richmond ignored vociferous penalty claims.
In the 64th minute Saunders bundled Samaras off the ball inside the box and this time Richmond did point to the spot.
Robson took responsibility but had his penalty dramatically saved by Ruddy with the Motherwell defence clearing the rebound.
In the 76th minute Samaras should have buried the chance given to him by Robson but his poor shot from 12 yards allowed Ruddy to grab the ball at full stretch.
Two minutes later Fortune replaced Fox as Celtic stepped up the pressure.
Seconds later, in the most dramatic fashion, the former Nancy striker took a McDonald pass at the edge of the box and drilled in a wonderful shot past the helpless Ruddy.
In the second minute of added time, with the game stretched, Fortune crashed a shot off the outside of the post but his contribution had already been made.