# Block Ahrefs Crawler User-agent: AhrefsBot Disallow: / # Block Semrush Crawler User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: SemrushBot-SI Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: SemrushBot-CT Disallow: / User-agent: SemrushBot-BM Disallow: / # Block Moz Crawler User-agent: rogerbot Disallow: / # Block Majestic Crawler User-agent: MJ12bot Disallow: / # Block Xenu Crawler User-agent: Xenu Disallow: /