An early Bondz N'Gala header was enough to beat Bury although Pompey fans were made to sweat as their team hung on to claim the victory in the end.
The match started at blistering pace with Bury attacking from kick-off and they might have taken the lead in the first minute but Shaun Harrad could only direct his header straight into the arms of Phil Smith who was standing in for Trevor Carson.
Pompey went straight down the other end and John Marquis headed Ricky Holmes' cross against the underside of the bar. Portsmouth fans jumped up in celebration but soon sat back down as the on-loan Millwall striker was flagged offside and the ball may not have crossed the line anyway.
However, there were no questions asked when N'Gala headed Holmes' corner past Brian Jensen to score his third goal in two home games.
The game was surprisingly open and full of strong challenges. Tom Soares went down easily under Yassin Moutaouakil's challenge and both Joe Devera and N'Gala had to also time their tackles to perfection to win the ball in the box as Bury began to gain the upper hand.
The Shakers continued that improvement into the second half and Smith had to make a fantastic reflex save by diving low to his left to prevent Harrad's header going in.
Whittingham's men were beginning to show nerves as Bury continued to pile forward and capitalise on some woeful passing and poor clearances to test Smith again. Thankfully the visitors were wasteful and blasted a couple of good chances over the crossbar.
The home fans were furious late in the game when referee Carl Berry blew for a foul on a Pompey player when they were about to instigate a four-on-one counter-attack.
Sonny Bradley was almost punished for a poor touch inside his own box but Anton Forrester could only drag his shot wide when he should have scored.
Jed Wallace could have put the result beyond doubt in the final minutes when he played a one-two with Holmes on the counter but could only find the side netting.
Pompey managed to see out stoppage time and when the final whistle blew the feeling around Fratton Park was relief more than anything. It's safe to say that if Bury weren't so wasteful then they would have at least taken a draw from the match.
Fans' reaction to this game seems to be that "it's good to win when playing badly" but we all know that if Bury got the goal they probably deserved then the "Whittingham out!" crew would have been sharpening their knives once again as he failed to react to the visitors' tactics once again and left it very late to bring on any substitutes to change the game.
It's hard to pick positives from that performance as Pompey were poor but at least Phil Smith didn't let us down and the defenders were committed enough to clear much of the danger at the last attempt.
However, Wycombe are unlikely to be as wasteful as Bury when they come to Fratton Park on Tuesday and so Pompey must improve if this unbeaten run is to extend.
Pompey Player Ratings
Phil Smith: 7 - Distribution was poor but made a couple of excellent saves.
Yassin Moutaouakil: 6 - Passing was woeful but tackling was much better.
Sonny Bradley: 6 - Solid but should have paid for late mistake.
Bondz N'Gala: 8 - Got the vital goal and a rock at the back.
Joe Devera: 7 - Seems to be improving.
Jed Wallace: 6 - Faded quickly. Not at his best.
Therry Racon: 6 - Got stuck in.
Johnny Ertl: 6 - Not his worst performance by any means.
Ricky Holmes: 7 - A threat whenever he had the ball.
Patrick Agyemang: 6 - Worked hard but not too influential.
John Marquis: 6 - Wasted chances but always put the Bury defence under pressure.
Source: Fratton Faithful