However, before the interval, Celtic striker Marc-Antoine Fortune scored to stem the tide.
It still looked a long way back for Tony Mowbray's side but the former Nancy frontman doubled his tally in the 67th minute to set up a tense finale and, in stoppage time, Paul McGowan, making his first start of the season, fired in the leveller from close range to keep Celtic in third place.
A Group C campaign which began with a defeat in the heat of Israel against Tel Aviv concluded in freezing conditions in Vienna but, if nothing else, Mowbray will be warmed by the fighting spirit if his players.
Aside from McGowan's inclusion, the Hoops boss had given rare starts to Willo Flood, Mark Wilson, Paul Caddis and Niall McGinn.
However, on a freezing cold Austrian night, the ploy to keep players fresh for Sunday's Clydesdale Bank Premier League game at Hearts backfired spectacularly as the visitors founds themselves a goal down before a minute had lapsed.
McManus surrendered possession at the back to Rapid captain Steffen Hofmann and the ball was moved on to Jelavic who drilled a low shot from the edge of the box past Artur Boruc, with just 33 seconds on the clock.
Celtic and their small band of supporters were stunned and were visited by more woe in the eighth minute after their offside trap was cut wide open.
Salihi cut the ball across the six-yard box for Jelavic, who had scored in the 1-1 draw at Parkhead, to knock the ball over the line from a yard out despite the attentions of Hoops defender Glenn Loovens.
It became even more embarrassing for the Glasgow side in the 19th minute when Jelavic returned the favour for his strike partner, turning Loovens inside out before setting up Salihi who slid the ball in from eight yards for number three.
The Parkhead side were handed a lifeline in the 23rd minute when Wilson's cross into the box from the left broke kindly for Fortune who turned and slammed a shot from 10 yards high into the net.
Moments later, with Celtic having at last found some belief, McGinn forced a decent save from Rapid's stand-in goalkeeper Raimund Hedl with an angled shot.
The game had settled down and it was the Hoops, inspired by Fortune's strong running, who were edging possession.
Seven minutes from the break, though, Celtic midfielder Landry N'Guemo gave possession away to Veli Kavlak in the middle of the park and, after brushing past Caddis, his shot was deflected past for a corner which the visitors survived, albeit unconvincingly.
But it was Celtic who ended the half stronger and a minute before the break Hedl had to dive high to his left to push McGinn's long-distance drive away from the top corner, with the visitors failing to take advantage of the corner.
Celtic began the second half brightly and within minutes of the restart Hedl made yet another fine save from McGinn, who had been set up inside the box by McManus, by tipping a low drive past the post.
The Parkhead side were back on the ascendancy but it began to look as if it was not McGinn's night as Hedl blocked another angled drive from the Northern Ireland international, this time with his foot, before making a more comfortable save from his left-footed drive moments later.
Rapid had lost their early momentum, perhaps understandably, and as Celtic chased, harried and probed, they struggled to make any inroads towards Boruc.
Celtic got their just rewards in the 67th minute when quick thinking by Flood at a free-kick sent McGinn racing down the right channel and when his cross came in to the middle Fortune had the simplest of tasks to score from barely a yard out.
Graham Carey replaced McGinn to make his Celtic debut with around 13 minutes left and still the Parkhead side pushed for the leveller as the hosts continued to deteriorate.
And in the first minute of stoppage time McGowan drove in from close range after Marc Crosas had chipped into the box to end a disappointing campaign on a positive note.