Loan striker Theo Robinson snatched promotion-chasers Hereford United full points with a stunning 89th minute winner.
The Watford hitman beat Bury keeper Jim Provett with a sweet strike just as it looked as if the hosts had grabbed a battling draw.
It was his fourth goal in as many games and the striker's tenth goal of the season.
Chris Casper's battling team had the better of the chances on a freezing cold afternoon in Lancashire and striker Andy Bishop could have a hat-trick if his finishing had been better.
But the Shakers missed out on their fourth home draw of the season as Hereford stretched their impressive unbeaten run to eleven matches in all competition as they stay second in the table.
It was also Hereford's first win at Gigg Lane since 1984, their last triumph a 4-1 win in March of that year.
The visitors started the brighter of the two sides and they went close to scoring in the seventh minute, but a superb last-ditch tackle from Andy Parrish spared the home team's blushes.
Two minutes later Steve Haslam headed off his own line after eventual goalscorer Robinson headed Sam Gwynne's right-wing cross goalwards.
At the other end broke and Glynn Hurst was unlucky to have a goal disallowed for offside.Nicky Adams' delightful right-wing cross was flicked on by Paul Scott and Hurst fired into the bottom right-hand corner, only to see the far side linesman waving his flag for the offside.
Bury went close in the 33rd minute. Ben Futcher met Adams' free-kick with a solid header, but Hereford keeper Wayne Brown saved well at the back post.
Bury keeper Provett was forced into action two minutes later. Toumani Diagouraga tried his luck from 25 yards out, but Provett saved well.
In the 38th minute Adams lost his marker on the right edge of the penalty area, but the youngster wasted a good chance, firing well wide with a right-foot shot.Simon Johnson had a shot from distance five minutes before half-time, but his shot didn't trouble Provett.
On the stroke of half-time Bishop should have scored, but from Adams' pinpoint cross from the left failed to hit the target from the near post.
Provett saved well from Steve Guinan in the first minute of the second half, as the striker tried his luck from distance.
Bury went agonisingly close to scoring themselves. Parrish played in Adams inside the box and his cross was met by Bishop, but the striker saw his powerful header saved by Brown at his back post.
Parrish crossed for Bishop in the 65th minute, but the centre-forward headed straight into the arms of Brown.
A minute later Adams fired over from just outside the box as the home side looked to make their dominance in possession count.
Gywnne headed wide as Hereford looked for the opener, before Bishop forced Brown to make a smart stop after a lovely pass from Scott.
Scott should have scored with ten minutes to go, but he fired wide when he should have hit the target.