Craig Beattie's cross went straight in to give the Baggies the lead but Micky Cummins levelled for the visitors.
Graeme Dorrans put the Baggies back ahead, but Pope levelled almost straight away.
The former Crewe man then put his side in front for the first time in extra time, but Beattie grabbed his second before Cox's winner.
The Baggies made nine changes to the side which beat Ipswich at the weekend and it was the visitors who had the first effort on goal, with Nicky Law's shot forcing Dean Kiely in to action.
Cox had a shot blocked and the returning Roman Bednar was off target with a header before the Baggies went in front in the 10th minute with a freak strike from Beattie.
The Scotland striker crossed from the left and it went straight through without a touch and nestled into the net to give him just his sixth goal for the Baggies in more than two years at The Hawthorns.
Long-range shots were the efforts of choice for the next quarter of an hour or so with both sides chancing their arms, although Beattie could have doubled his tally from a Marek Cech corner, but headed over the top.
And the Millers drew level just before the half-hour when Jamie Green sent a cross in from the left and Cummins headed home in a crowd of players.
Beattie was denied by Andy Warrington, but Rotherham had gained confidence from their leveller and were playing some good football against their Championship opponents.
But five minutes into the second half the Baggies got their second after excellent work from Cech.
The Slovakia full-back won the ball in his own half and burst down the flank before squaring for Dorrans to lash home his second goal for the club.
Celebrations were short-lived, though, as the Millers levelled within a minute. Law swung in a corner and Pope, a £150,000 summer signing from Crewe, superbly headed in his first goal for the club.
Cox saw his shot saved by Warrington but the Millers were looking dangerous at the other end, especially from set-pieces.
Pablo Mills had a shot blocked before Kiely had to deny Harrison as the Millers looked for a shock, but Cox came close to a winner in normal time when his shot was cleared off the line by Marc Joseph.
Extra-time therefore beckoned and Cox was involved in the first chance of the extra period when he found Dorrans but Warrington did enough to keep out his well-struck effort.
But the Millers took the lead in style when Pope's cross was cleared back out to him and he sent a superb volley past Kiely.
Again though, the lead was short-lived when Beattie also grabbed his second, collecting the ball, turning inside his man and smashing in.
Cox fired just wide in a one-on-one with Warrington towards the end of the first period of extra time, but he made no mistake five minutes from the end when he fired home from Chris Wood's pass.
Kiely had to make a fine save with the last touch of the match to keep out Law's free-kick, but it was the Baggies who progressed.