Brown has been under pressure following a poor start to the season which sees City lie third bottom of the table after 11 games, with just eight points.
It follows a dreadful run at the end of last season, which only saw the Tigers achieve safety thanks to Newcastle's final day capitulation.
The club are refusing to comment on rumours Brown has left his post at the KC Stadium but leading bookmakers have reacted to the speculation by taking down their next manager to go market. Sky Bet are already offering odds on the next permanent Hull City manager with Alan Curbishley and Gareth Southgate already installed as the joint 5/1 favourites.
A 0-0 draw against Portsmouth at the weekend saw a number of fans vocalise their frustration, although chairman Paul Duffen has remained ever supportive of Brown in public.
Amid mounting speculation over the Hull manager's future, former striker Dean Windass called for Brown to "get moved on".
Windass, who left Hull in January after scoring the play-off goal that sent the Tigers to the top flight, told talkSPORT magazine: "I think sometimes you need a fresh change. Phil's done a fantastic job since he's been there but I think it's time that he should get moved on and bring somebody else in."
Windass, questioning whether the club can stay up this season, added: "It's going to be difficult. They're not getting results and they're struggling to score goals at the moment.
"They need a 10-15 goal-a-year striker to get away from the bottom of the table. They've got a little mini-league at the moment and it's important they win games against teams in and around their league."
Brown gave an interview to the Hull Daily Mail on Tuesday insisting he would not quit the club, although rumours have started spreading about his future.
There have also been concerns about the club's long-term financial position, stemming from accounts being published covering the year ending July 2008.
In that 12 months Hull made a loss of nearly £10million as they won promotion to the Premier League via Windass' goal at Wembley.
This does not include any figures from the club's first season in the top tier of English football, and the directors remain confident the next set of results will be much more positive.