In an evenly contested first half, both sides failed to take their chances.
Rovers' Darryl Duffy had the first opening but his shot was charged down by Hereford goalkeeper Adam Bartlett. Visiting defender Darren Jones also headed a corner from Kenny Lunt over the top from close range.
Rovers' Steve Elliott failed to find the target with a powerful header while a shot on the turn from Hereford's Marc Pugh flashed inches wide of the upright.
Just before the break Rovers' new goalkeeper Rhys Evans, who has signed a one-year contract, made a fine save to stop Lee Morris putting Hereford in front.
Play continued to switch from end to end in the second half but Duffy missed a gilt-edged chance for Rovers when he shot wide with only Bartlett to beat before the match was settled by the shoot-out.