How to Allow User-Triggered AI Fetchers (ChatGPT, Claude, Perplexity)

Why It Matters

Blocking user-triggered fetchers prevents prospective customers from using AI tools to analyze your pricing pages, product documentation, or articles, causing immediate user friction.

• ChatGPT-User - User-directed fetcher for OpenAI ChatGPT.

• Claude-User - User-directed fetcher for Anthropic Claude.

• Perplexity-User - User-directed fetcher for Perplexity AI.

Robots.txt Explicit Allow Configuration

Add explicit allow directives for user fetchers to prevent accidental blocking via general wildcard rules.

User-agent: ChatGPT-User
Allow: /

User-agent: Claude-User
Allow: /

User-agent: Perplexity-User
Allow: /

Robots.txt May Not Apply

Because these visits are started by a person, OpenAI and Perplexity say their user-triggered fetchers may not follow robots.txt rules. To keep a page private from AI tools, protect it with a login rather than relying on robots.txt.