The York-born player joins the Black Cats for an undisclosed fee and has penned a three and a half year deal on Wearside.
Kilgallon becomes the ninth signing to arrive on Wearside under the reign of manager Steve Bruce; the eighth on a permanent basis.
Bruce said: "Matthew is a very talented defender and has been coveted by a number of top flight clubs recently so I'm very pleased to have brought him here.
He will add to our options across the back ahead of some very important games. I'm sure he'll prove to be a great signing for the club.
Kilgallon began his career at Leeds United, making 80 league appearances between 2004 and 2007, while he also made three appearances for West Ham United during a short loan spell in 2003.
In January 2007, he moved across Yorkshire to join Neil Warnock's Sheffield United, where he would amass over 100 league appearances for the Blades. Kilgallon has represented England at Under-21 level five times.