A replay was the very least the Conference side deserved after they outplayed League Two side Vale for long periods of an entertaining cup tie.
The visitors must have wondered how they were behind at half-time after creating chances at regular intervals in a very impressive display.
But Vale took the lead when Adam Yates tried his luck from 30 yards out which evaded the dive of Chris Day, who should have done better.
Day made amends by pushing a Marc Richards header against the crossbar two minutes after the interval before Stevenage responded with a further spell of pressure.
And while Vale stemmed the tide for a while, Stevenage threw everything into a late barrage which ended with Griffin heading in Mitchell Cole's cross from six yards.