The 30-year-old agreed to terminate his contract with the Blues following their relegation from League Two and has signed a two-year deal with the Millers after passing a medical.
Ellison scored nine goals last season in his second spell with Chester and has also played for Leicester, Stockport, Lincoln,
Hull and Tranmere.
"I've always had faith in my own ability and I always knew I would find a Football League club who wanted to sign me," he said.
"I didn't play many games towards the end of the season but Rotherham obviously saw something they liked and asked Chester whether they could talk with me."