The Sunderland captain has been booked 47 times and sent off on five occasions so far in his senior career but defends his tough-tackling methods as nothing more than an attempt to win the ball.
Red faced: Cattermole is giving his marching orders by referee Anthony Taylor
He said: 'I feel like they [the referees] are out there for me at the minute.
'Maybe it's my stupid mistakes in the past that have got me that reputation, so I can't sit here and blame everyone else.
'I have to get rid of that reputation, but it's so hard at the minute when I find myself getting a succession of soft yellow cards.'
Cattermole was substituted immediately after receiving a yellow card in the Black Cats last game before the international break against West Brom by manager Steve Bruce.
And the 23-year-old, who has represented England at every age group bar the senior side, believes he has only ever made one bad tackle in his career.
Foul play: Cattermole feels Baggies midfielder Steven Reid tried to get him sent off
'If you put every foul I've ever made up on the television, I think there'd be one tackle which was actually a bad one,' said Cattermole.
'That was against Scott Parker when I was playing for Wigan and I've spoken to him since to apologise about it.
'That's the only bad tackle I've made in my career - all the rest have been genuine attempts to win the ball, not nasty attempts to hurt someone.'
The former Middlesbrough and Wigan midfielder is not only angry at the officials but players as well who he feels jump at the chance to get him sent off.
After his substitution against the Baggies Cattermole was fuming that half of the opposition, in particular Steven Reid, were trying to make him see red.
'The tackle I got booked for wasn't a foul, simple as that.
'But it's twice it's happened now with Steven Reid. He's six foot four and he rolls around like a girl, simple as that.
'It's twice it's happened and it's a shame. People moan about the foreigners, but he's a British lad and he's acting like that. I was booked for that challenge, and then the second one was on him again.'
Sunderland 2 West Bromwich 2: Great Dane! Bendtner saves Bruce's baconCattermole will bounce back from being dropped, insists Sunderland boss BruceO'Shea insists there is no need to panic despite Sunderland's slow start