Every AI crawler, every host
Lanesra reads the robots.txt on your site and on your docs and blog subdomains, and shows what each AI crawler may fetch from your homepage and key pages.
Enter your domain. Lanesra checks your robots.txt, firewall, page tags, llms.txt and sitemap against every AI crawler, from GPTBot to ClaudeBot, and shows which engines may train on, index and visit your site.
Free, no signup. Checks ChatGPT, Claude, Gemini, AI Overviews, Copilot, Perplexity and Mistral.
See which crawlers are let in, what each one is there for, and what keeps out the ones that decide your answers.
Lanesra reads the robots.txt on your site and on your docs and blog subdomains, and shows what each AI crawler may fetch from your homepage and key pages.
Each crawler is sorted by what it does: training a model, building a search index, or opening a page a user asked about. Blocking training keeps your content out of models without costing you answers.
We request your site as each AI crawler and as a browser. When the crawler is turned away and the browser is not, a firewall or bot-protection rule is blocking it, whatever robots.txt says.
Find a missing or broken llms.txt, a robots.txt that errors, and a sitemap that is missing or stale, before they cost you a citation.
Spot noindex, nosnippet and noarchive tags that keep your pages out of Google's AI Overviews and Microsoft Copilot, even when every crawler is allowed.
Lanesra drafts an llms.txt from your own pages, measured against the questions you want to win, ready to put on your site.
Connect Lanesra to Claude, Cursor, n8n, and any tool in your stack through MCP and API. Pipe visibility and citation data straight into your own workflows.
Start Here
Check your domain for free, or connect your logs for a full record of every AI crawler that visits.