Bora-Hansgrohe rider Sagan was punished by the commissiares after he shoved Wout van Aert (Jumbo-Visma) in the final 50 metres of the stage.
Sagan had initially finished second on the stage, but was relegated last place in the peloton for dangerous sprinting, which has made his target of winning the green jersey a lot more difficult.
The former world champion has now shared his thoughts ahead of stage 12.
“What can I explain?,” Sagan said. “I found myself on the right side of the road. I just saw the little space and I tried to pass, that’s all.
“I don’t see the movement that I did as being dangerous. I had the speed and I wanted to pass.”
Speaking to Eurosport Sagan also said he was worried about the advertising poles on the barrier, which stuck out into the road.
He said: “I saw the barriers were pretty close and in the middle of the barrier there was a pole with the brand on it.
“If I kept going I was going to touch it with my handlebars, but I didn’t think it was that bad or dangerous.”
Some fans suggested that Sagan had made the manoeuvre to avoid a selfie stick that a spectator was holding over the barrier, but Sagan said he didn’t see the selfie stick and was more worried about the barrier.
As a result of the incident the commissaires stripped Sagan of his second place, handed him a 500CHF (£412) fine and had 13 points deducted in the points classification.
Wout van Aert, who said he tried to speak to Sagan after the finish but only received strong words in response, was also punished for his reaction.
After the line, Van Aert raised his middle finger towards Sagan which then resulted in a 200CHF (£165) fine.
Sagan is now 68 points off the lead in the green jersey competition.