Brown helped the club escape relegation from the Barclays Premier League in May but has eight points from 10 games in the current campaign, and they are above only Portsmouth and West Ham in the table.
Windass, who left Hull in January after scoring the play-off goal that sent Hull to the top flight nearly 18 months ago, told talkSPORT magazine: "I think sometimes you need a change. Phil's done a fantastic job since he's been there but I think it's time that he should get moved on and bring somebody else in."
Windass, questioning whether the club can stay up this season, added: "It's going to be difficult. They're not getting results and they're struggling to score goals at the moment.
"They need a 10-15 goal-a-year striker to get away from the bottom of the table. They've got a little mini-league at the moment and it's important they win games against teams in and around their league."