What is llms.txt and How to Create One for Your Site

Why It Matters

As AI assistants increasingly synthesize web information, providing a lightweight, markdown-formatted directory helps language models navigate site information efficiently without parsing full HTML visual page layouts.

Format Example (/llms.txt)

File Placement & Structure: Host the file at /llms.txt using structured markdown headers, concise blockquote summaries, and clean links to primary documentation.

# Insight Bench
> Insight Bench provides website audit fixes and AI visibility optimization tools.
## Core Resources
- [Learn](https://insightbench.com/learn): Free guides to fixing common website problems.
- [How we score](https://insightbench.com/how-we-score): The checks, weights and score bands behind the Insight Score.

Maintenance Routine

Keep llms.txt synchronized with major product updates and new documentation releases to ensure AI bots access accurate site links.

Set Expectations

llms.txt is a proposal from 2024, and no major AI company has confirmed that its crawlers use it yet. It takes minutes to create and does no harm, but it won’t replace crawlable pages, clear content and structured data.