Killie netted through Frazer Wright, Conor Sammon and Alexei Eremenko to move off the foot of the Scottish Premier League with a third win of the season.
"I'm very pleased with the performance and the collective spirit," Paatelainen said.
"I'm delighted we got a victory in the end. It was excellent."
An unmarked Wright converted Jamie Hamill's free-kick in the second minute of first-half stoppage time before two goals in three second-half minutes sealed victory for the visitors.
A long ball from Hamill down the right allowed Sammon to sneak around the Hearts defence and beat Marian Kello with a well-placed shot after 80 minutes.
And Eremenko curled a third into the top corner two minutes later as Hearts' defence failed to deal with a long ball forward.
Paatelainen was pleased Killie have finally been rewarded for their strong displays this term.
He added: "Our performances have been good in general so far this season so I'm not overly concerned.
"I know if we convert a fraction of the chances we create, we can score enough goals to stay in this league and that's the target for this season."
Killie have now scored eight goals away from Rugby Park this term, two more than in the whole of last season.
"We've always created loads of chances to score goals, we haven't put them away," Paatelainen added.
"Today the boys were fantastic. Three goals away from home is tremendous."
Eremenko is on loan from Ukraine side Metalist Kharkiv until the end of the season and Paatelainen believes the Finland international is the most technically gifted player in the league.
"He's a quality player," he said. "I've not seen a better passer of the ball than him in this league and it's amazing that a player like that plays for Kilmarnock.
"Hopefully he shoots more because he can score more goals for us, but he also creates lots."
It was not a good day for former Kilmarnock manager Jim Jefferies or Kevin Kyle, who moved to Tynecastle from Rugby Park last summer.
Jefferies anticipated Killie would prove a difficult proposition and had no complaints.
He said: "They deserved to win the match. We didn't use the ball well enough and when we did, the quality of the ball wasn't good enough.
"A free header in a set play in injury time in the first half gives them something to hold onto and then you worry about them hitting you on the break and that's exactly what they did.
"I can't believe the goal we virtually teed up for the boy (Eremenko) to hit and he's taken advantage of it and he hit it well."
Hearts meet Hibs in the Edinburgh derby next Sunday and Jefferies believes his squad will be motivated in training next week despite today's setback.
"What a game to bounce back, away at Easter Road next week," Jefferies said.
"It's the perfect game to get everyone lifted and a win there will go a wee bit to helping suffer the disappointment of today."