But Pilkington then missed his spot-kick and Jim Goodwin also saw his penalty saved as Chesterfield went through to the quarter-finals.
Chesterfield were in front after 57 minutes when Donal McDermott delivered a superb cross to the back post and Talbot headed past Alex Smithies, but Huddersfield levelled seven minutes later.
Tommy Lee saved Jordan Rhodes's header but Pilkington followed up to volley in the rebound. However, Talbot struck again when he smashed in Mark Little's cross after 74 minutes.
Bowery fired low into the bottom corner in the 89th minute but Huddersfield took the tie to penalties through a Pilkington free-kick and a close range finish from Clarke in the 90th minute.
However, Chesterfield kept their nerve to progress, with Jamie Lowry, Jack Lester and Dan Gray scoring as Pilkington and Goodwin were made to pay for their misses.