The dispute centres around the role which Fulham claim Richards played in England striker Peter Crouch's £9million move from Portsmouth to Tottenham two years ago.
Fulham refused to accept a High Court judgment that the dispute with Richards should be settled by arbitration, but three Court of Appeal judges on Thursday dismissed the Premier League club's appeal after a hearing in London.
However, Fulham will now launch an appeal to the Supreme Court in an attempt to have the case heard in open court.
"Fulham will be seeking permission from the Supreme Court to appeal this decision since it considers that a point of law of general importance is involved and that it has a good prospect of success," the club said in a statement.
"The decision by the Court of Appeal affects not only the world of football but potentially every company in the land which has a shareholders' agreement or set of articles of association with an arbitration clause."
Fulham had complained that Richards helped "facilitate" Crouch's transfer to Tottenham, but the Premier League chairman denies any impropriety.
Fulham, who had also bid £9million for Crouch, claimed Richards had acted as an "unauthorised agent" in breach of the game's regulations.
Last year they asked Mr Justice Vos to restrain Richards from participating in transfer negotiations or to order that he should "cease to be chairman" of the Premier League.
The judge refused, following a High Court hearing, and said the dispute should go to arbitration.
Fulham tried to persuade the Court of Appeal to overturn the ruling.
But Lord Justice Rix, Lord Justice Longmore and Lord Justice Patten today upheld Mr Justice Vos' decision and unanimously dismissed the appeal.
Judges heard that the Premier League had concluded that Sir David's role in the Crouch transfer was "legitimate".
A Premier League inquiry had found that he "played a part in 'mediating"' between Portsmouth and Tottenham but did not "broker the transfer".