Skip to main content
AI crawlers don’t run JavaScript, so an analytics tag never sees them. Crawler tracking runs on your server or CDN instead: a few lines of code send each crawler request to SeoSorted. It powers the AI crawler visits number on your dashboard and the Pages AI crawler can’t open card on the AI Visibility page.
Only crawler requests are sent. Visits from people are never sent, and the code never slows your pages down: it sends in the background and ignores errors.

​
Set it up

1

Turn on tracking

Open Project Settings → AI Crawler Tracking (or Set up tracking on the AI Visibility card) and click Turn on tracking. You need to be a workspace admin.
2

Pick where your site runs

Choose your platform. The code shown already contains your website ID.
3

Add the code and deploy

Paste it where the screen says, then deploy. If someone else manages your site, use Copy instructions for your developer and send them the text.
4

Check it works

Run the test command from the setup screen:
The status turns to Connected within a few seconds. Test requests never count as crawler visits.

​
Which option to pick

Why Next.js can’t find missing pages: proxy.ts runs before the page renders, so it can’t see whether the page answered “not found”. Visits are still tracked. If your domain runs through Cloudflare, use the Cloudflare option to get both. Hosted website builders (Shopify, Wix, Squarespace, Webflow, Framer) don’t allow server code. They work only if your domain is proxied through Cloudflare (orange cloud), where the Cloudflare option runs in front of them. Shopify’s storefront can’t be proxied this way.

​
What the status means

​
Troubleshooting

  • Check the change is deployed to the live site, not only saved.
  • Check the website ID in the code matches the one on the setup screen. Rotating the key stops the old ID immediately.
  • A normal browser visit isn’t sent. Use the test command, which pretends to be a crawler.
  • Run the second test command under Still waiting? Troubleshoot. It skips your site and calls SeoSorted directly. If that connects but the site test doesn’t, the code isn’t running on your site yet.
We check each visit claiming to be GPTBot, ChatGPT, Perplexity or Googlebot against the crawler’s published addresses, and leave fakes out. If your server sits behind a load balancer or proxy, it may be sending its own address instead of the visitor’s. On Express, keep app.set("trust proxy", true). Elsewhere, forward the x-forwarded-for header.
The card re-checks every page live before listing it, so a redirect you’ve added drops it from the list on the next check. Open the card’s details and click Re-check.
Some sites answer every unknown address with a normal page instead of “not found” (a soft 404). Crawlers then never get a “not found”, so nothing can be listed. Make unknown addresses return a real 404 status.
WP Rocket, LiteSpeed Cache and similar serve cached pages without running PHP, so some visits aren’t sent. Missing pages still show up, because “not found” pages aren’t cached. For full coverage, use the Cloudflare option.

​
Turning it off

Turn off tracking on the setup screen stops SeoSorted accepting data from your site. What was recorded stays, and turning it back on resumes with the same website ID. You can then remove the code from your site.