Platform

See which AI bots read your site

Server logs show what AI crawlers fetch. Send a copy of each request to Citeflare and see which bots visit, which pages they read and what status they get back.

Citeflare crawler analytics screen with demo data

How it works

How crawler analytics works

  1. 01

    Create a project token

    Each project gets a private crawler token for the ingest endpoint.

  2. 02

    Add a snippet

    Paste a Cloudflare Worker, Next.js middleware or call the API from your own server. It forwards request details without changing the response.

  3. 03

    Bots are classified

    User agents are matched against known AI crawlers such as GPTBot, ClaudeBot and PerplexityBot.

  4. 04

    Browse the log

    See the latest 50 visits with their path, and which pages are read and which return errors.

Capabilities

Everything in one place

Known AI bots

GPTBot, ClaudeBot, PerplexityBot and other AI crawlers, identified by user agent.

Cloudflare Worker

Copy-paste snippet that reports requests and passes them through untouched.

Next.js middleware

A small middleware for sites hosted on Vercel or anywhere Next.js runs.

Ingest API

POST request details from any server, CDN or log pipeline with your token.

Status codes

See which pages crawlers read successfully and which return an error status.

Visit timeline

See each AI company’s visits per day, and when they stop.

In depth

What you can do with it

Visit log

The latest bot requests, in one log

The log lists the latest 50 AI crawler visits with their path, flagging error statuses, so you can see what each bot reads.

  • Time, bot and path per request, with error statuses flagged
  • Latest 50 visits
The latest bot requests, in one log: the Citeflare app with demo data

Coverage

Which pages get read, and which do not

If a bot never reaches the pages you want cited, there is nothing to cite. See your top 20 crawled pages, which ones AI answers also cited, and which return errors.

  • Most-crawled paths and how many bots read each
  • Errors served to crawlers
  • Compare bots against each other
Which pages get read, and which do not: the Citeflare app with demo data

Setup

Connected in minutes

The Worker snippet runs at the edge, forwards a small record of each request, and returns the original response. Nothing is added to the page.

  • Works with Cloudflare, Next.js or the API
  • One token per project
  • No change to page content or speed for visitors
Connected in minutes: the Citeflare app with demo data

Metrics

What we measure, defined Every number has one definition, so you can check it against the data.

Bot hits
The number of requests a given AI crawler made in the period.
Pages crawled
Distinct paths requested by AI crawlers.
Error rate
Share of crawler requests that returned any status other than 200.
Last seen
The most recent time a bot visited, per path.

Related

Works with the rest of the platform

Prompt tracking

Track the prompts your buyers ask AI. Citeflare scans ChatGPT, Gemini and Perplexity daily and Claude weekly, recording mentions, position and sentiment.

Citations

See which domains and pages AI engines cite, split into owned, competitor, editorial and user-generated sources, including Reddit and YouTube.

Content agents

Turn a gap into a brief, an article and a published page. Citeflare’s content agents write, schedule and publish to WordPress, Ghost, Shopify, HubSpot, Contentful, Webflow, Framer or a webhook, and refresh existing pages with a diff.

Faq

Questions, answered

Is crawler analytics on every plan?

It is included on Growth and Agency. Starter does not include it.

Does the snippet slow my site?

The Worker reports in the background and returns the original response as normal. Test it on a staging route first if you want to be sure.

Which bots are recognised?

Known AI crawlers such as GPTBot, ClaudeBot and PerplexityBot, matched by user agent. Anything unrecognised is not shown as an AI bot.

Can the user agent be faked?

Yes, user agents can be spoofed. Treat the log as a strong signal, not proof.

What do you store?

Bot name, user agent, path, status code, IP and time for each request. No page content and no visitor data beyond that.

See what AI says about you. Your first scan runs the day you sign up.

Monthly or annual plans. Cancel any time.