Two minutes later Williams turned provider with a low cross for Jeff Hughes to flick home from close range.
Jo Kuffour added the third goal in the 34th minute, shooting into the roof of the net from Stuart Campbell's quickly taken free-kick.
Hughes hit the crossbar with a delicate chip with 20 minutes remaining as Rovers produced an incisive performance.
Swindon's best chance fell to substitute Alan O'Brien but he produced a weak finish when unmarked in the box five minutes after the break.
Rovers defender Steve Elliott cleared off the line from Alan Sheehan just before half-time, but it was a rare moment of encouragement on an off day for Danny Wilson's visitors.