The Tykes pulled themselves level when poor defending from Rob Jones allowed second-half substitute Daniel Bogdanovic to fire home.
But with a draw looking likely, the ball fell inside the box to Paul Hayes who curled beyond Luke Steele to claim a 2-1 victory and all three points for his side in time added on.
"That's what it's all about," said a delighted Adkins. "You come down and don't leave the game early. That's the message.
"It was a great finish by Paul Hayes, we built up some great pressure at the end there with a series of corners and we're delighted we got the victory.
"Great credit to the players, the character that was on show following the defeat at the weekend was brilliant.
"The first half was a very nervy one and was perhaps a reflection of that.
"Gary Hooper scored a great goal from a Matt Sparrow cross and had a number of one-on-ones that he could have capitalised on, but at least he gets himself into those positions.
"Well done to the lads, with 10 minutes to go Barnsley got an equaliser and we kept going. We thought we could win the game and we've gone and done it."
United moved up four places to 17th in the Championship following their victory and Adkins revealed his side had targeted Tuesday night as one of the games they felt was winnable.
He added: "We've looked at the run of games that we've got and we identified the home games that gave us a good opportunity and we've got to go and capitalise on them.
"It was pretty in a different way where it was competitive. Both sides were competitive and we got the win in the end."
Barnsley boss Mark Robins, whose side have now lost three consecutive games for the first time since he took over as manager, thought his players should have taken their chances.
He said: "If you don't take your chances, then unfortunately you're not going to get a point.
"We created a fair few, missed them and so we got punished."
Robins also revealed he had not envisaged his side losing at Glanford Park.
He added: "I expected to get at least a point from this game, if not the three. I'm really gutted we didn't get a point but on the balance of play I think we should have got all three."