Known AI bots
GPTBot, ClaudeBot, PerplexityBot and other AI crawlers, identified by user agent.
Platform
Server logs show what AI crawlers fetch. Send a copy of each request to Citeflare and see which bots visit, which pages they read and what status they get back.

How it works
01
Each project gets a private crawler token for the ingest endpoint.
02
Paste a Cloudflare Worker, Next.js middleware or call the API from your own server. It forwards request details without changing the response.
03
User agents are matched against known AI crawlers such as GPTBot, ClaudeBot and PerplexityBot.
04
See the latest 50 visits with their path, and which pages are read and which return errors.
Capabilities
GPTBot, ClaudeBot, PerplexityBot and other AI crawlers, identified by user agent.
Copy-paste snippet that reports requests and passes them through untouched.
A small middleware for sites hosted on Vercel or anywhere Next.js runs.
POST request details from any server, CDN or log pipeline with your token.
See which pages crawlers read successfully and which return an error status.
See each AI company’s visits per day, and when they stop.
In depth
Visit log
The log lists the latest 50 AI crawler visits with their path, flagging error statuses, so you can see what each bot reads.

Coverage
If a bot never reaches the pages you want cited, there is nothing to cite. See your top 20 crawled pages, which ones AI answers also cited, and which return errors.

Setup
The Worker snippet runs at the edge, forwards a small record of each request, and returns the original response. Nothing is added to the page.

Metrics
Related
Track the prompts your buyers ask AI. Citeflare scans ChatGPT, Gemini and Perplexity daily and Claude weekly, recording mentions, position and sentiment.
See which domains and pages AI engines cite, split into owned, competitor, editorial and user-generated sources, including Reddit and YouTube.
Turn a gap into a brief, an article and a published page. Citeflare’s content agents write, schedule and publish to WordPress, Ghost, Shopify, HubSpot, Contentful, Webflow, Framer or a webhook, and refresh existing pages with a diff.
Faq
It is included on Growth and Agency. Starter does not include it.
The Worker reports in the background and returns the original response as normal. Test it on a staging route first if you want to be sure.
Known AI crawlers such as GPTBot, ClaudeBot and PerplexityBot, matched by user agent. Anything unrecognised is not shown as an AI bot.
Yes, user agents can be spoofed. Treat the log as a strong signal, not proof.
Bot name, user agent, path, status code, IP and time for each request. No page content and no visitor data beyond that.
Monthly or annual plans. Cancel any time.