Skarz allowed Kevin Lisbie to cross for Odejayi to net the late winner, leaving Huddersfield without a win in five matches.
Clark said: "We were strong defensively and we stuck to the gameplan, but the sending off affected us and the goal came from an individual error.
"We've been kicked where it hurts, but I have told the players that they have nothing to be ashamed of.
"The luck is not with us at this moment in time, but I'm sure it will turn. There were so many good things from us.
"We certainly deserved something from the game. We kept the gameplan and there were some creditable performances."