An entertaining game was settled in Derby's favour by a late winner from Gary Teale after a George Boyd penalty cancelled out a fifth-minute opener from Miles Addison.
It was hard on the visitors but Ferguson said: "Once we start believing in ourselves we can definitely compete at this level.
"I was pleased with our performance for the last hour but two set pieces has cost us which is very frustrating.
"There is no point going into a lead thinking you are going to make up the numbers. We have to believe we can get to the Premiership.
"I think we will be fine if we play like we did for the last hour. Everyone might think it's mental and mad to say we can get into the Premiership but I know what I have got in the dressing room and I want to take Peterborough into the Premier League.
"A lot of my players were playing non-league football two years and I do believe some of them will play at the top level. I did not think that they deserved to lose the game and I felt for them."
Derby manager Nigel Clough was happy to give the home supporters an opening-day victory to cheer in what he thought was a great advertisement for the Coca Cola Championship.
He said: "It's nice to send the supporters home with a win on the first day which is something they have not experienced for seven years.
"We want to be positive and create chances and for the majority of the game we did just that.
"It was certainly the right result for us and I thought the honesty and endeavour shown by both sides was an absolute credit to the Championship.
"Once we did not get that second goal they came back and put us under an awful lot of pressure.
"It was almost as though it was an end-of-season game with both teams going at it hammer and tongs - the mystery was how it only finished 2-1."
Derby had started last season by losing at home to newly-promoted Doncaster but this time they were in front in the fifth minute.
Paul Green swept in a cross from the left and when Jake Buxton returned the ball, Addison bravely stooped to head past Joe Lewis.
It was all Derby in the opening 20 minutes but Peterborough gradually got a foothold in the match and when Green was forced out of the action at half-time with a bruised foot, the visitors found the space to create openings.
They carved out four good chances in the space of five minutes but Aaron McLean wasted two of them, Dean Moxey made a great saving tackle on Craig Mackail-Smith and Dean Frecklington was denied by Stephen Bywater.
But in the 83rd minute, a mix-up between Bywater and Paul Connolly ended with the Derby keeper bringing down Shaun Batt and Boyd put away the penalty.
That should have been enough to earn Peterborough a point but three minutes from the end, Lewis failed to cut out a cross and after Addison headed against the bar, Teale got the final touch in a crowded six-yard area.