The Terriers looked set to spring a surprise when they were 3-1 up early in the second half courtesy of a Theo Robinson double and yet another goal from summer signing Jordan Rhodes.
But the Magpies hit back and Kevin Nolan put them through to the third round with an 84th-minute winner.
Clark said: "I have told the players they have done me proud - I thought they were magnificent. Obviously we were a little disappointed in the manner in which we conceded some of the goals, but we will talk about that later in the week.
"I thought some of our play at times was of the highest quality and I thought the pace of the game was very high at times.
"I think it will be difficult to find a better cup tie in the country this week. I am in the game to win and obviously we are disappointed to lose, but I am a very proud manager."