The O's host Southampton on Saturday and Williams is hoping for a big improvement against another of the so-called 'bigger' clubs.
"We've really got to believe in ourselves when we go to big grounds," said Williams.
"We said we were going up to Huddersfield to get a result. We have got to believe that. It seemed that as the game went on, we lacked that belief."