The Hammers are the only Premier League team not to register a point from their opening four matches and are bottom of the table.
They will travel to Stoke without manager Avram Grant and defender Tal Ben-Haim, who are observing the Jewish holy day of Yom Kippur which involves fasting and prayer over the course of Friday night and Saturday.
And despite their recent good record at the Britannia Stadium, where they had won four successive matches until last season's 2-1 defeat, Walters is confident West Ham's difficulties will continue.
"West Ham haven't had a win yet and are struggling and Stoke is a very difficult place to come to," said the £2.75million summer signing from Ipswich.
"They struggled last year and haven't started well so it hasn't surprised anyone.
"I'm not surprised to see them struggling because anyone can and I'm sure their heads are down at the moment and it won't get any better against us I don't think.
"They will be scrapping for everything they can but I'm not sure how they will be with their manager not being there and being away from home - it's not a good place to come to, the Britannia.
"We have to make the stadium like a fortress, giving it everything in our home games and trying to pick up what we can away and the fans play a major part in that."
The Potters recorded their first win of the season with a home victory over Aston Villa on Monday.
Walters believes they will now be able to build on the confidence gained from that result after a testing start to the season which pitted them against two of last season's top four in champions Chelsea and Tottenham.
"It is important to go in to Saturday's game with that belief," he said.
"No-one likes to be on no points and so the more we can get now will create a gap and a little cushion (to the bottom of the table).
"You always want to get the first win of the season early doors and now we want to push on.
"The start we had was not the best but Chelsea away was always going to be difficult to pick up points.
"We've got to start picking up more points now and we have some good games coming up against West Ham, Fulham, Newcastle, Blackburn and Bolton.
"We're not just playing the stronger teams like Chelsea away so those are games we will be looking to get something from."
Stoke assistant manager Dave Kemp has urged the players to use their dramatic injury-time victory over Villa to put a run of results together.
"Confidence is a big thing in football and wins are what help that," he said.
"We have to get away from the bottom half of the table before we start looking at the top half.
"As we have done since we have been in the Premier League we have to edge ourselves forward step-by-step."