The Stamford Bridge club have been embroiled in the fall-out from the Mark Clattenburg affair, with the club having accused him of using "inappropriate language" towards John Obi Mikel last weekend, which has come hot on the heels of the John Terry racism scandal.
"We've got off to a very good start this season and I don't think we have got enough credit for the start we have had," said Bertrand.
Saturday's 1-1 draw at Swansea, where Chelsea were denied victory by a late Pablo Hernandez strike, saw Roberto Di Matteo's side surrender top spot in the Barclays Premier League table.
"It's all been negative stuff," Bertrand added. "With the draw today we are a bit disappointed but we just look forward to the next game."
The fixture in south Wales was the final instalment of Terry's four-match domestic ban for racially abusing QPR defender Anton Ferdinand.
Di Matteo is unsure of whether to throw the former England international back into the side for the Champions League meeting with Shakhtar Donetsk in midweek, but Bertrand admits the return of such an experienced defender can only help Chelsea.
He said: "A player like JT would be a loss to any club in any league. But it has not affected us too much, thankfully, because we have very good players to fill in. Gary Cahill has been outstanding every time he has come in.
"But JT could still play for any team so it will be a massive boost for us when he does come back."
Source: PA
Source: PA