marcus.li ~/llms-txt
built with claude no signup no filesystem access needed

the lab

llms.txt Generator

Yoast SEO's llms.txt feature writes a file straight into your WordPress root, which a lot of hosting quietly blocks. This does the same job a different way: it reads your public sitemap, you pick what to feature, and you get a finished file to upload yourself. Works for any site, not just WordPress.

why this matters

More questions are getting answered by AI tools reading a summary of a site instead of crawling all of it, and llms.txt is the emerging way to hand them a short, curated list of what actually matters instead of leaving them to guess from a raw sitemap. Not having one doesn't get a site excluded, but it does mean AI tools are working with less than the owner intended. Worse is thinking you have one when you don't, which is exactly what happens when Yoast's file generation fails silently on restrictive hosting: nothing looks broken, because nothing errors, there's just nothing there.

step 1 · find your sitemap

Loading...
how this works
  1. Your site's robots.txt is checked for a Sitemap: line first (what Yoast and most SEO plugins publish there), falling back to /sitemap_index.xml and /sitemap.xml if needed.
  2. If it's a sitemap index (the normal WordPress/Yoast setup), the individual sitemaps underneath it are read too, up to a reasonable cap.
  3. Only pages you actually select get fetched for a title and description, nothing is scraped in bulk.
  4. Everything about your site read here is already public. Nothing is stored, this server keeps no database.

What an llms.txt file contains

An llms.txt file is a plain text file at the root of your site that gives AI tools a short, curated summary of what the site is and which pages matter most. This generator writes it in the usual format: a title, a one-line description, then sections of links, each with a short note. A sample, with a made-up site:

# Example Plumbing Co.

> Residential plumbing repair and installation in Burnaby, BC.

## Services

- [Drain cleaning](https://example.com/drain-cleaning): Same-day clearing of kitchen and bathroom drains.
- [Water heaters](https://example.com/water-heaters): Repair and replacement of tank and tankless units.

## About

- [About us](https://example.com/about): Who we are and the areas we serve.

You choose which pages go in, and you can edit the site name, the description, and each page's title and description before you download the file.

How to generate and upload an llms.txt file

  1. Enter your site's address and the generator finds your sitemap, checking robots.txt first and then the standard sitemap locations.
  2. Pick up to 25 pages to feature. The filter box helps on larger sites.
  3. Fetch titles and descriptions for the pages you selected, then edit anything that doesn't read the way you want.
  4. Click Generate, then download the file.
  5. Upload it to your site's root folder, in the same place as robots.txt, using FTP or your host's file manager. When it's up, yoursite.com/llms.txt should show the plain text.

Is an llms.txt file worth adding?

llms.txt is a proposed convention, not a standard every AI tool reads, and having one doesn't guarantee your pages get cited. It's cheap to publish, though, and it keeps your own short summary of the site in one place instead of leaving AI tools to piece it together. It works alongside robots.txt, which is the file that controls which bots may crawl you. To see which AI bots your robots.txt currently allows, use the AI crawl checker.

Frequently asked questions

Why does Yoast SEO's llms.txt feature fail on my site?

Yoast generates llms.txt by writing a physical file into your WordPress root folder. A lot of hosting, especially managed WordPress hosts, blocks PHP from writing files outside the uploads folder for security. On those hosts the feature fails, often silently.

How does this tool get around that?

It doesn't touch your server at all. It reads your site's already-public sitemap.xml, lets you pick which pages to feature, and hands you a finished llms.txt file to upload yourself, the same way you'd upload any other file.

What is llms.txt anyway?

A proposed convention, similar to robots.txt, that gives AI systems a short, clean summary of a site's most important pages instead of making them guess from a full crawl.

Does this only work for WordPress sites?

No. It works with any site that has a public sitemap, including hand-coded sites and other platforms. The Yoast problem is the common reason people look for it, but the tool doesn't depend on WordPress.

How many pages can I include, and where does the file go?

You can feature up to 25 pages at a time. The finished file goes in your site's root folder, next to robots.txt, and it needs to be named llms.txt.