Plummer, on loan from Bristol City, rifled home his second penalty in four days, having scored from the spot against Portsmouth in the Carling Cup on Tuesday.
Vale did everything but score in the first half when they forced six corners, with Louis Dodds heading the first against the Hereford crossbar after three minutes.
Hereford, however, struck in a counter attack when winger Mark Pugh raced past two defenders before curling a close-range shot past Chris Martin.
But the visitors levelled four minutes into the second half when Dodds rifled a low shot past goalkeeper Adam Bartlett from 15 yards.
Vale struck again four minutes later when Robert Taylor drilled a 25-yard shot on to the crossbar before the ball dropped down over the line.
But Hereford levelled 11 minutes from time when Plummer drilled his spot-kick past goalkeeper Martin after Richard Rose had been brought down by a Vale defender.