The chance was created by substitute Andy Williams, who teed up Kuffour 10 yards out and saw the striker beat Dean Brill with a confident right-footed finish.
Defences had dominated until then, with Rovers creating what few chances there were. Mark Wright sliced wide from the best of them after 13 minutes.
Oldham's best opportunity fell to Nick Blackman 13 minutes from time, but he shot straight at Mikkel Andersen from 10 yards.
Oldham goalkeeper Brill also made good saves from Stuart Campbell and substitute David Pipe.
Reuben Hazell was a rock at the back for the visitors and unlucky to finish on the losing side. It is now five wins from seven games for Rovers, their best start under coach Paul Trollope.