Microsoft Clarity Now Flags Bots That Ignore Robots.txt

Microsoft Clarity’s free Bot Analytics dashboard now detects and displays bot requests that violate a site’s robots.txt rules, showing violations as a percentage of total bot activity. Site owners can filter by bot operator, name, and requested URLs to compare compliant crawlers against rule-breakers. The feature requires enabling AI Visibility settings and using a supported CDN (Fastly, CloudFront, Cloudflare, Azure Front Door, Akamai) or the latest WordPress plugin. It only reveals that violations occurred, not why, giving site owners a free way to monitor AI crawler behavior.