The Gateshead-born striker blasted home six minutes from time to seal the win after Kevin Nolan had converted substitute Danny Guthrie's 61st-minute cross.
But Chris Hughton's men were made to work all the way to the whistle as Karl Duguid's 50th-minute equaliser, which cancelled out Steven Taylor's opener, threatened to hand Argyle some reward for their 818-mile round-trip to Tyneside.
On the 10-year anniversary of the late Sir Bobby Robson's first home game as Newcastle boss - an 8-0 victory over Sheffield Wednesday - his latest successor sent his side out looking for a response.
Wednesday night's 2-1 defeat at Blackpool was the Magpies' first
of the season and knocked them off the top of the table, and they wasted little time in setting out their stall when Taylor stooped to meet Geremi's corner beyond the far post and power home a header on six minutes.
Peter Lovenkrands was denied by keeper Romain Larrieu's fingertips after Nolan had flicked on Geremi's 13th-minute cross, but Argyle rallied and Cillian Sheridan tested Harper at his near post with injured full-back Danny Simpson limping impotently in his wake.
Simpson was swiftly replaced by Ryan Taylor, but the visitors were enjoying their best period of the half, thanks in part to the carelessness of their hosts in possession.
Hughton's men enjoyed a massive stroke of luck five minutes before the break when Jose Enrique was robbed by Jamie Mackie inside his own penalty area and Taylor turned his driven cross on to the post and the rebound was hacked away.
Larrieu kept out Ryan Taylor's 43rd-minute effort at point-blank range, but there was little doubt Newcastle needed to rediscover their rhythm when they returned after the break.
But the visitors were back in it within five minutes of the restart when Sheridan was allowed to make ground down the right and cross for Duguid to fire past Harper.
The Magpies keeper pulled off a superb diving save to tip away Gary Sawyer's curling right-foot shot three minutes later as the home side wobbled alarmingly.
Hughton sent on Guthrie and Nile Ranger for Joey Barton and Lovenkrands in a bid to regain a foothold in a game which was rapidly slipping away from his team, and the changes paid dividends almost immediately.
Within two minutes of his arrival, Guthrie latched on to Carroll's flick and crossed for Nolan to head home unopposed at the far post.
Duguid fired just wide from David Gray's cross with 15 minutes remaining, but Carroll, who also hit the bar in injury time, made sure with an emphatic 84th-minute finish to calm the nerves of the bulk of a crowd of 42,898.