Robots.txt file for LLMs


Did you know there is basically a robots.txt file for LLMs? There is a proposal for a llms.txt file that will help LLMs parse your site. SEO is dead… well it’s evolving and the old way of doing SEO is done and gone. These LLMs look to take the place of the search engines we have relied on so heavily for the past few decades.

One thing I found interesting was that the format appears to be a modified version of the MarkDown Syntax. Looking at the examples on llmstxt.org it looks very human readable.

My question for you: What are you doing to make sure your product/service can be indexed by the new wave of LLM tools?

Let me know if you want me to do a deeper dive on this or perhaps make a video. ~Cheers