On Tuesday, police charged 11 people in connection with the incident at Portsmouth, and while insisting there was no room in the game for such abuse, James admitted fans still have the right to give 'stick'.
"As long as it is not racist or homophobic, give stick, give stick," said James. "It is football. You can't all be happy chaps going around saying everyone's wonderful.
"As Isaac Newton said, there is an equal and opposite reaction to every action. It is possible to get distracted from the objective of getting a result. Loss of emotional control is a loss of objectivity.
"Ours has to be winning the game. Our emotions have to go out of the window."
The fact that Portsmouth face former manager Harry Redknapp and ex-player Jermain Defoe, coupled with the return to Spurs of Campbell, and the fact that Portsmouth are led by former Arsenal favourite Tony Adams, all adds to the lively mix.
Peter Crouch, who formed a dangerous partnership with Defoe before his £15.75 million return to Spurs, admitted the game could be affected by so many equations.
"You'd like to say it's just another game," said Crouch, who scored his 13th goal of the season in Portsmouth's 2-0 FA Cup third-round replay win at Bristol City on Tuesday.
"Harry did so fantastically well for Portsmouth but he's moved on and so has Jermain Defoe, so there will be a lot of stuff off the pitch. But both teams are down there fighting and we certainly need to get the result.
"Jermain made up his mind quite early that he wanted to go to Tottenham. I think he feels more at home there, he still has a house there and was travelling into Portsmouth.
"He's happier at Spurs, so the club will go on without him and I am sure we'll have success.
"I don't think I've got anything to prove to Harry. I know he rates me as a player. He's bought me a few times and always had confidence in me and I've got a lot of respect for Harry because he's done a lot for my career.
"A lot of people will talk about the game being Harry against Portsmouth but we'll just concentrate on getting the points.
"There's a few that weren't in the team under Harry who will want to show that they should have been playing, so there might be a bit of that, but first and foremost we just want to win for Portsmouth."