Rafael van der Vaart's controversial first-half effort - he appeared to use his arm to bring the ball down - opened the scoring for Spurs before Aaron Ramsey equalised six minutes after the break.
Walker then scored with a superb strike which swerved at the last moment to beat Gunners keeper Wojciech Szczesny.
Walker told Sky Sports 1: "It doesn't get much better than that. I just picked the ball up and hit it.
"I had one of them against Fulham for Aston Villa so I thought why not take a chance.
"I just thought why not have a go. If you don't buy a lottery ticket you don't win the lottery.
"Arsenal are a good team and we found it difficult at times but we dug deep and got the result that we wanted."
Spurs boss Harry Redknapp praised his young defender, who has spent time on loan with QPR and Aston Villa.
"He's a great attacking full-back and he loves to get forward," Redknapp said.
"It was a good strike from Kyle. He's done ever so well and he's had a good start to the season."
And Redknapp was pleased his tactics came to fruition as Arsenal's poor start to the season continued.
"They had a lot of possession in the first half. It was difficult. They played with three in midfield with one up and so they were always going to hold sway in terms of possession.
"I played two strikers so I went for a very attacking team and we got the goal before half-time.
"To be honest I nearly made a change before half-time to thicken us up in there but I thought we'll see how we go.
"They got off to a better start in the second half but we upped it and got better and stronger and in the second half we had all the chances."
The win moves Tottenham into sixth place in the Premier League while Arsenal are now in 15th.