Crawler reads

Crawlers do not run JavaScript, so no browser script sees them. With the proxy in your middleware, every request from a known crawler is recorded with its name, the page and the response status, and sorted into a class:

Class	Examples
ai_training	GPTBot, ClaudeBot, Applebot-Extended
ai_answers	ChatGPT-User, Claude-User
search	Googlebot, Bingbot, Applebot
other	other known crawlers

The dashboard’s AI & crawlers page shows reads per class, per crawler (with error counts) and per page. The same data is at GET /v1/sites/{id}/stats/crawlers and in the crawlers MCP tool.

Crawler reads do not count as events for your plan.