Aston Villa striker Agbonlahor and Manchester City defender Richards joined up with Stuart Pearce's squad towards the start of the week but have now returned to their clubs.
"Both players arrived with injuries and have had treatment with the medical staff, but they have not responded to treatment," said an Football Association spokesman. "They have now been sent back to their respective clubs as a precautionary measure."
Pearce has not called up a replacement for the Norway friendly on Friday evening in Sandefjord.
City goalkeeper Joe Hart and defender Nedum Onouha, Middlesbrough centre-back David Wheater and Aston Villa midfielder James Milner are among the players with experience who will travel to Norway.
Aaron Lennon was moved up to the seniors and Theo Walcott was ruled out through injury, meaning Pearce does not have the strength of squad he wanted for the double-header.
Pearce still wants to pick his strongest possible squad for the European Championships in Sweden this summer, after the last tournament when Agbonlahor and David Bentley controversially withdrew.
In a move to pre-empt any repeat, Pearce has written to every member of his provisional squad for the tournament, telling them not to book any holidays until after the tournament.
"I thought it very important this time to go into it and make sure that the club managers and especially the players were aware of it," Pearce said.
"I used to like to know what was in front of me as a player, in regards to booking holidays, and all the players know full well that anything that is going to be booked is going to be after 29 June, the final date."