Gunn was only relieved of his duties four days ago following a humiliating 7-1 defeat at the hands of Lambert's Colchester.
The 40-year-old former Celtic midfielder had guided Colchester to two wins from their opening two games of the new League One season, and has impressed the City hierarchy enough to have been offered the job.
"It's a big club and there is no way we should be where we are," he told Norwich's official website.
"We want to get promoted but we certainly understand there are going to be some challenges if that is going to become a reality."
Lambert took over at Colchester in October 2008, having previously managed Livingston and Wycombe. He is charged with guiding Norwich back into the Championship following their relegation at the end of last season.
Ian Butterworth oversaw Saturday's 1-1 draw with Exeter and will also lead the team against Brentford on Tuesday evening while Lambert watches from the stands.
"We're absolutely delighted to announce Paul as our new manager," said City chief executive David McNally.
"He was our number one choice and fulfils all the criteria the board had set."
The two clubs are discussing a compensation package.