29 Ways AI Search Finds You

AI search can only cite what its crawlers can reach and read. This list covers the structure behind that, from robots.txt to the dates in your sitemap.

It is the working list behind my Agent-Ready Website Audit, published so you can run it yourself. If you want the model side of how agents pick pages up, read my post on how agent skills work.

Access for AI crawlers

1. Allow the main AI crawlers in robots.txt. GPTBot, ClaudeBot, PerplexityBot, Applebot-Extended, Meta-ExternalAgent and Amazonbot each need an explicit Allow rule. Open /robots.txt and search for every one of those names.
2. Keep the CDN and the firewall from blocking those user agents. A robots.txt Allow changes nothing when Cloudflare or your firewall answers with a challenge page or a 403. Request one page per bot user agent and confirm you get the same 200 a browser gets.
3. Keep llms.txt at the site root, and current. It should answer what the site is, who runs it and which pages matter most, the way you would brief a new client. Mine sits at /llms.txt; check yours after every offer change.
4. Point robots.txt at the XML sitemap. That Sitemap line is how crawlers find pages internal links miss. Open robots.txt, load the sitemap URL and confirm it parses.
5. Answer plain fetches with 200 and real HTML on your key pages. Soft 404s, interstitials and login walls on money pages hide the copy you want quoted. Fetch your top five pages with curl and read the status code and the body.

Identity and structured data

6. Mark up the Organization in JSON-LD. Name, URL, logo and sameAs give the model one entity to attach your pages to. Validate the block in the Rich Results Test and confirm the sameAs URLs match your real accounts.
7. Mark up the WebSite and tie it to the publisher. The WebSite node names the property in search features and connects it to the Organization or Person behind it. Check that the @id values in both nodes reference each other.
8. Give the author a Person entity with sameAs. Service pages and articles should resolve to a real person with live LinkedIn and GitHub profiles. Open the schema, click every sameAs link, and confirm each profile carries the same name.
9. Keep the sameAs profiles alive and consistent. A dead or mismatched profile undercuts every claim the markup makes. Visit each URL and compare the name, the photo and the role with your own site.
10. Match BreadcrumbList to the visible breadcrumb trail. Schema that disagrees with the path shown on screen gives crawlers two stories about the same page. Put both side by side on a deep URL.

Canonical and index hygiene

11. Give every page a canonical that points at itself. Self referencing canonicals stop parameter variants and protocol clones from splitting attention across copies. View source and confirm the canonical equals the URL you would paste into a chat.
12. Keep og:url equal to the canonical, byte for byte. When those two disagree, shares can point at the wrong version of the page. Diff the two strings on every template.
13. Take noindex off pages you want cited. The rule can live in the meta robots tag or in the X-Robots-Tag response header, and either one hides the page. Inspect both on every service page, article and tool.
14. Give every page a unique title under 70 characters. The title is still the clearest label a crawler reads. Export the titles from your CMS or sitemap, then sort for duplicates and for length.
15. Give every page a unique meta description under 160 characters. Assistants often repeat the description as the summary. Run the same export and fix the duplicates first.
16. Serve one host version and redirect the rest. Pick https and one hostname, then 301 the alternates into it so every signal lands in one place. Try the other spellings of your home URL and watch where they end up.

Content an agent can quote

17. Render the body on the server. When view source shows an empty shell and the text appears only after JavaScript runs, many crawlers never see the copy at all. Open the page with JavaScript off and read what is left.
18. Answer the page's core question in the first screen. Agents lift short declarative passages, so the definition, the price or the promise should sit in plain sentences near the top. Read the first paragraph on its own and ask what it proves.
19. Use one H1 per page and keep the heading order sane. A single H1 with nested H2 and H3 elements gives the text a shape a parser can follow. Check the outline in a headings map.
20. Match FAQPage markup to the visible FAQ, byte for byte. When the schema drifts from the page copy, you get the kind of mismatch search engines reject. Diff the JSON strings against the rendered text on the live page.
21. Put real prices in Product and Offer markup on product pages. If the visible price is 49 and the schema says 99, the contradiction is the fact the model repeats. Compare the schema price with the price a customer sees.
22. Set max-snippet:-1 and max-image-preview:large. These robots meta values let search engines quote long passages and show large previews of your pages. Confirm the tag on every page you want cited.

Speed and trust

23. Write lastmod dates that match the last editorial change. A timestamp stamped by the build system claims a rewrite on every deploy and drowns the real signal. Compare the date on each sitemap URL with the visible updated date on the page.
24. Show visible dates that match datePublished and dateModified. When the page says one date and the JSON-LD says another, the model has two stories to pick from. Compare the date shown on screen with the schema on every article.
25. Add a skip link and label every form field. Keyboard users and crawlers that parse forms both depend on them. Tab through the page and complete the main form without a mouse, then check that every field announces its purpose.
26. Keep the whole site on HTTPS with no mixed content. An http image or script on an https page trips warnings and can stall the fetch. Load your key templates and watch the browser console for mixed content.

Measurement

27. Record AI crawler hits in your logs or analytics. You cannot fix access for a bot you never see. Filter your server logs or CDN analytics for the user agents you allowed in check one.
28. Tag AI referrals so they land in a named channel. Traffic from chatgpt.com and perplexity.ai often sits in direct or other. Open the referrer report for the last thirty days and add a channel for the sources you find.
29. Ask the assistants the questions you want them to answer. Once a month, run your money queries in ChatGPT, Perplexity and Google AI Mode and write down what they say about you. The gap between that answer and your own copy is the next fix.

Visibility earns the visit. If the leak is in what happens to the lead after that, the Lead Routing Contract Builder is the next tool.

FAQ

What is the AI visibility checklist?

A list of 29 structural checks I run before I trust a site to show up in AI answers. Each check has a reason and a way to verify it.

Does llms.txt help AI search find my site?

It gives assistants a curated map of your pages and speeds up how fast they understand what you sell. It matters after the crawlers are allowed in, so pair it with open robots.txt rules and clean schema.

How often should I rerun the checklist?

Run the full list after a redesign, an offer change or a platform migration. In between, a monthly check of citations and crawler traffic is enough.