The two sides remained deadlocked after 90 minutes, although the hosts squandered numerous chances of an upset.
Both Andros Townsend and Adrian Patulea missed simple opportunities and Kitson made them pay with a fantastic long-range strike shortly after the extra 30 minutes had began
His 35-yard half-volley soared over the head of Orient goalkeeper Glenn Morris to end the stubborn resistance of Geraint Williams' team.
The Premier League outfit fielded a team of reserves against with midfielder Amdy Faye being the only player to keep his place in the side from Saturday's draw at Birmingham.
Stoke were quickly onto the attack and in the ninth minute Kitson fired just over the crossbar from 20 yards.
Moments later, the City striker was wide of the target with a volley after Carl Dickinson's cross had found him lurking by the penalty spot.
PressureBut Orient were not to be outdone and they came forward in the 13th minute with Ryan Jarvis sending a 20-yard effort over the crossbar.
Moments later a mistake by Stoke's Ryan Shotton allowed Townsend to fire an angled drive towards the near post but Steve Simonsen held the shot at the second attempt.
In the 19th minute, Orient almost took the lead when Townsend again tested Simonsen's reactions with a left-foot shot that the City goalkeeper did well to parry to safety.
Orient lost goalkeeper Jamie Jones with a knock on the half-hour and he was replaced by substitute Morris.
Morris was called upon moments later though when Tom Soares pulled the ball back for Richard Cresswell to hit a shot towards the bottom corner of the net.
However, Morris somehow managed to get his fingertips to it and push it away for corner.
The keeper had to be at his best again in the 38th minute when Kitson tried his luck with an angled drive towards the near post.
Morris dived low to his right to prevent the ball from sneaking in as Stoke briefly raised the pace of their game.
Orient should have gone ahead just before half-time when a free-kick by Charlie Daniels spun across the six-yard box and was somehow missed by Adrian Patulea.
ChancesThe hosts began where they left off after half-time with good work from Patulea presenting Townsend with another great opportunity to open the scoring, but the Orient midfielder sent his rising effort over the crossbar.
Patulea wasted another great chance in the 47th minute when John Melligan's cross found the Orient striker unmarked at the far post. But Patulea's header was weak and lacked accuracy as he sent it wide of the target from close-range
Orient continued to cause Stoke problems in defence and Patulea's neat flick-on in the 58th minute was met by Tamika Mkandawire.
However, the Orient defender had his back to goal and his attempted header fell neatly into the arms of Simonsen.
Stoke should have snatched a winner in the 83rd minute when Pugh's corner was met by the head of Kitson, who sent his header crashing into the crossbar before Melligan cleared.
However, Kitson did settle the tie in the third minute of extra-time when his terrific 35-yard effort found the top corner of the net.