Jonathan Tehoue pulled a goal back for Orient four minutes from time to set up a tense finish but Tranmere were able to hang on.
"We did not do the basics in the first half and thoroughly deserved to be losing the game at that stage," said Williams.
"The second-half performance was much better. We created a number of chances and should have scored more goals.
"We looked a poor side in the first half and a decent side later but it is still very disappointing and I am a bit annoyed. We did not apply ourselves properly and we conceded poor goals. We lost it in the first half and we need to improve."