Speaking in his rest day press conference on Monday, the UAE Team Emirates rider said that he understood people asking questions "because of what happened in the past".
Asked if he understood the scepticism and the questions, the 24-year-old said: "Yeah, I understand. I always get this question every year now at the Tour so I don't see any difference this year to other years. We are riding [at our] best I must say. Every stage we go full and it's a hard one.
"But yeah, I understand people, that they ask questions, because of what happened in the past. Some people don't get get over it. And I completely understand them."
The Slovenian currently lies second overall after two weeks of racing at the 2023 Tour de France, just 10 seconds behind his big rival Vingegaard, who rides for Jumbo-Visma.
He and Vingegaard have been breaking records up the race's biggest climbs almost everywhere it has gone.
Asked on Sunday after an explosive stage 15 whether some fans are right to be doubtful given the history of doping within the sport, Vingegaard replied: “To be honest, I fully understand the scepticism. We have to be sceptical because of what happened in the past. Otherwise [if people weren’t] it would just happen again.
“In that way I fully understand all the questions we get about it. The only thing I can say is that I am not taking anything, but to be honest I am happy there is a bit of scepticism about it.
“We are going faster, quicker, maybe even quicker than back then. I think it’s a good thing.”
Vingegaard added that the reason why times are faster than ever is because of technological and scientific advancements. “The food, the material, the training, everything is different,” he said.
“Once again, it’s always good to be sceptical about it, or to at least think about it.”
With the pair so tight on time at this year's race, there has been speculation that the Tour could come down to who is on top mentally, but Pogačar was keen to stress that he did not know who had the advantage psychologically, and it would just come down to legs.
"I think we both just sleep and race in the moment," he said. "We don't look so much at the history of duels and battles but for sure we push each other in all the ways to improve ourselves and improve the teams. I don't know who would have an advantage in the psychological battle. He has his own mind, I have my own mind.
"I must say that I I enjoy this battle against him. This is the third year in a row now so it's a good duel and I respect this battle a lot."
Racing resumes on Tuesday with a 22.4km time trial from Passy to Combloux, with over 600m of climbing, before over 5,000m of climbing on stage 17 to Courchevel, including the Col de la Loze. However, Pogačar thinks the race might go deeper.
"Stage 20 can really be the hardest stage of the Tour in the end," the UAE rider said. "We will see how the situation is. If it's still tight, tied, and we need to gain some time, a lot can happen. You need to attack early and that makes the stage super hard."
Thank you for reading 20 articles this month* Join now for unlimited access
Enjoy your first month for just £1 / $1 / €1
*Read 5 free articles per month without a subscription
Join now for unlimited access
Try first month for just £1 / $1 / €1