Which AI crawlers does it check?

How the verdict is reached

The audit does not visit your site twenty times pretending to be each bot. It fetches your robots.txt once and reads it the way a crawler would, then tests the audited path against each of the 20 tokens in turn. For every token it records whether the rules allow that path or disallow it. The single crawlability check rolls those 20 verdicts into one row, and the breakdown behind the row names each crawler and what it got.

The thresholds are fixed:

• No crawler blocked: pass. A missing robots.txt passes too, since a file that does not exist blocks nobody.
• One or two blocked: warn, and the breakdown says which.
• Three or more blocked: fail.

Why the list is this long

Every engine sends its own crawler, and several send more than one. OpenAI alone announces GPTBot, ChatGPT-User, and OAI-SearchBot. A robots.txt written with Google in mind, which is most of them, can let Google through and shut every other name out without anyone noticing, because nothing on the Google side breaks. That is the failure this check exists to catch: a page that ranks fine and is invisible to the engines that build answers.

What the check cannot see

The verdict comes from the rules, not from a live visit. A firewall that turns crawlers away by user agent leaves no trace in robots.txt, so it will not show here. If that same firewall turns our auditor away, the run ends unreachable and says so, which is its own signal. In Managed GEO, robots rules are one of the fixes our team ships and confirms with a re-run.