Brown is reportedly running out of time to keep hold of his job following the Tigers' 2-0 defeat at Burnley on Saturday, which left the club in the relegation zone.
The Tigers have also won just three of their last 33 matches in the league, but Barmby believes Brown's achievements in securing the club's place in the top flight should afford him more time to turn the club's season around.
"You have to remember that Phil Brown took this club into orbit," said Barmby.
"Now it is up to us to make sure that he stays in a job. He is a good manager. But it is a results business and Phil Brown knows that more than anyone.
"He has been in the game a long time. He has broad shoulders, but it is up to us to get the results to keep him here.
"What he has done for this club is amazing - the first manager to get us in the top flight after 104 years, so we certainly want him to stay."
FrustratedBarmby also believes key decisions want against Hull at Turf Moor, with Burnley taking the lead through a debatable penalty and then Geovanni having a free-kick disallowed.
The veteran midfielder added: "They were massive decisions. For the penalty, Mears tried to hit it with his right, but caught it with his left and that meant he was off-balance. I don't know who gave it, the linesman or the referee.
"But it was a decision that left us very frustrated and disappointed. It turned the game in their favour.
"We got going in the second half and, in spells, we played the best we have for a while. And the Geovanni free-kick is something we all want to be in a ground to see.
"It was a fantastic free-kick and everyone was cheering. But then the referee blows and I don't know what it was for. For us, it was another incredible decision.
"Then they get the second goal and you know it's not your day."