Despite falling behind early on to a Felipe Morais goal, Pompey rallied and drew level when Jon Harley placed a fine shot into the bottom corner. But the hosts netted a fortuitous winner when Morais' cross from the left somehow evaded everyone and nestled in the far corner to leave Appleton annoyed.
"It was a frustrating game for us," he said.
"I don't mind losing a game of football but I do mind losing the way we lost this in terms of their winning goal.
"It was an aimless ball and no-one's touched it and we should have dealt with it - that's probably the biggest thing I'm annoyed at more than anything.
"We didn't come out the traps in the first half but we probably had the better chances and I was happy with the performance in the second half as we moved the ball better."