Nigerian international Sam Sodje headed the south Londoners into an early lead before a 25-yard thunderbolt from Anthony Pilkington levelled the scores after 38 minutes.
Izale McLeod then scored the winner within minutes of the restart with his second league goal of the campaign, leaving Charlton with the points and the Terriers with praise for their display.
But Clark said: "It's no consolation. We have lost the game. It's points I want not plaudits. We deserved at least a point and possibly even more.
"We got done by two set pieces. When you don't do that right you get punished and we have twice."