The Spireites were tipped for promotion but for a second season running failed to make the play-offs leading to protests from some supporters during their 2-0 defeat to Bradford on the final day of the season.
Chesterfield chairman Barrie Hubbard said: "Lee has had two full seasons as manager, in which the club has come close to getting promotion but we feel the time is now right to make a change. We thank Lee for all his efforts and wish him and his family all the best for the future."
Richardson, who was appointed in March 2007 after five years as assistant manager, said: "I want to thank my staff and the players of the last two seasons for all their efforts. I also wish to thank chairman Barrie Hubbard and the Board of Directors for giving me the opportunity to manage Chesterfield. I recognise it's time for us all to have a change."
Richardson added: "In thanking the fans of Chesterfield FC for their support, I would also like to wish everyone involved at the club the very best for the future."