Midfielder Sawyer joined the Shrimpers before the start of the season and was due to stay with the League One club until January.
However, he was left out of Steve Tilson's squad for last Saturday's game at Carlisle United after a breach of club discipline for failing to attend a team meeting.
He had made six substitute appearances for the Shrimpers in the league this season, but has now asked to go back to Chelsea.
A club statement read: "Southend United have accepted Lee Sawyer's request to return early to Chelsea from his loan deal."
The Shrimpers have received some good news, however, after Grant signed a two-and-a-half year extension to his contract to tie him to Roots Hall until 2012.
Skysports.com revealed last week that Grant was on the brink of extending his deal, which had been due to expire at the end of the season.
The former Chelsea youngster told the club's official website: "It's good to have got it all out of the way early as we have been talking for a little while now and I can just concentrate on my football now."