The 20-year-old midfielder was axed from the Shrimpers squad which travelled to Carlisle on Saturday following a breach of club discipline.
A club statement read: "Southend United have accepted Lee Sawyer's request to return early to Chelsea from his loan deal."
Sawyer's spell was supposed to run until January.
The Shrimpers have received some good news, however, after midfielder Anthony Grant signed a two-and-a-half year extension to his contract to tie him to Roots Hall until 2012.
"It's good to have got it all out of the way early as we have been talking for a little while now and I can just concentrate on my football now," said the 22-year-old ex-Chelsea man.