CiterankBot

CiterankBot is Citerank's web crawler, used to audit pages for AI search visibility signals. It only crawls pages that users explicitly submit for analysis.

Verifying CiterankBot

Anyone can copy a user agent, so CiterankBot proves its identity with a cryptographic signature instead. Every request it makes is signed with Web Bot Auth (HTTP Message Signatures, RFC 9421, Ed25519), the standard Cloudflare and others use to recognise signed agents.

Each request carries Signature-Agent, Signature-Input and Signature headers. The signature covers your host name (@authority) and the Signature-Agent header, is tagged web-bot-auth, names its key by RFC 7638 thumbprint in keyid, and expires five minutes after it is made. A request that says it is CiterankBot but carries no valid signature from a key in the directory is not from us.

CiterankBot runs on cloud infrastructure without fixed IP addresses, so verify it by signature rather than by IP.

What CiterankBot does

• Fetches page HTML to check schema markup, meta tags, and structured data
• Reads robots.txt and llms.txt for AI crawler access signals
• Checks response headers for security and caching configuration
• Measures page load performance signals
• Compares content between standard and bot user-agents (cloaking detection)

What CiterankBot does NOT do

• It does not crawl sites autonomously or build a link index
• It only fetches URLs that users explicitly submit in the Citerank dashboard
• It does not store or republish your page content
• It respects robots.txt directives

Blocking CiterankBot

To block CiterankBot from fetching your pages, add this to your robots.txt:

Note: blocking CiterankBot will prevent Citerank users from running audits on your pages.

Contact

Questions about CiterankBot? Email us