--- title: Module 7 — LLMs.txt, robots, and "recommended reading" | GEO Academy by BotPass canonical_url: https://botpass.io/academy-modulo-7/?botpass_lang=en last_updated: 2026-07-08T15:43:06+00:00 source: BotPass API entity: botpass.io author: BotPass Crawler language: en categories: fact_check: - "Facademy-modulo-7%2F)[Create free account](https://botpass." - "Facademy-modulo-7%2F) 🎓asteriskAcademy Level." --- Module 7 — LLMs.txt, robots, and "recommended reading" | GEO Academy by BotPass GEO Academy – Modulo 7: LLMs.txt, robots y lectura recomendada – BotPass — WordPress GEO Plugin for AI Visibility ### 🧭This module turns your architecture (Module 3) and your access policy 💡 (Module 6) into a practical guide so agents know asteriskwhat to readasterisk asteriskfirstasterisk. It is the module that closes the discovery loop: you have organized your house, decided which doors to open, and now you put a sign at the entrance that says "start here." Without that sign, a common and silent failure occurs: AI reads secondary pages, skips canonicals, and answers with incomplete or outdated versions of your truth. ### If you do not tell the agent what matters, it improvises There is an idea that runs through the entire Academy and reaches its most practical expression in this module: an agent that does not know what matters on your site improvises. And you already know where improvisation leads, because you have seen it in every previous module: citing pages that are not canonical, mixing definitions from different versions, pulling prices without their conditions. The agent does not improvise out of malice or laziness; it improvises because nobody has told it where to start, and when in doubt it takes the first thing it finds, which is rarely your best content. This module's solution is to stop waiting for the agent to guess and tell it explicitly what to read. To do that you build three assets that work together: asteriskllms.txtasterisk, your prioritized reading list; the asteriskAI Indexasterisk, your reference hub for humans and agents; and asteriskreading pathsasterisk, recipes that go from a specific intent to the sequence of pages that resolves it. All three share the same purpose: replacing the agent's guesswork with a guide you control. ### Why guiding reading is the last link in discovery It is worth situating this module on the map of everything you have done so far, because it is the piece that makes the previous work pay off. In Module 3 you built architecture so routes exist. In Module 6 you decided which parts of those routes stay open. But a route that exists and is open can still go untraveled if the agent does not know it should travel it first. Guiding reading is the link that connects potential with real use: the difference between having a perfectly organized library and also putting a librarian at the door who says "for what you are looking for, start on this shelf." There is also an important expectations nuance worth fixing from the start, which we will develop further below: these assets guide, they do not compel. An agent can ignore your llms.txt just as a visitor can ignore the sign at the entrance. But experience shows that a clear, prioritized, well-maintained guide consistently tips the scale in your favor, and that accumulated bias across thousands of queries is exactly what you are after. ### Where BotPass fits: closing the loop with evidence This module has a nice particularity: it is where you check whether all the work from previous modules is paying off. BotPass is how you validate that your "recommended reading" actually works and is not just a well-intentioned document. After publishing your llms.txt and AI Index, go back to BotPass and check whether bots start consuming your intended asteriskcanonical sources of truthasterisk more consistently than before. That shift of consumption toward your canonicals is the signal that the guide is working. And conversely: use BotPass to detect agents that still land on outdated pages despite your guide. Each of those cases is an actionable diagnosis —maybe a redirect is missing, maybe internal links are not clear enough, maybe a clean view is needed— and you fix it with the tools from modules 3 and 6. The cycle closes on itself: guide, measure, correct, measure again. ### llms.txt: the editorial sitemap for AI Let us start with the first asset. The best way to understand llms.txt is to think of it as an asteriskeditorial sitemapasterisk designed for AI. A traditional sitemap lists all your URLs for search engines, without hierarchy or criteria, with a goal of exhaustiveness. llms.txt is the opposite in spirit: short, prioritized, and centered on your canonicals. It does not try to list everything, but to point to the few things that truly matter. If a sitemap says "here is everything there is," llms.txt says "if you are only going to read a few things, let them be these." Writing it well is an exercise in discipline more than effort. Start with one to three lines describing what the site is and who it is for. Follow with your three to five main hubs, the same ones you defined in Module 3. Add your sources of truth —pricing, documentation, policies—. Include, if applicable, your canonical definitions as a glossary, drawing on the Entity Sheet from Module 5. And close with notes for agents such as "prefer the canonical URLs above" or "if a clean view exists, read it first." The temptation to add more and more will be constant, but every extra line dilutes the important ones: brevity is not a limitation of the format, it is its main virtue. ### AI Index: a hub for humans and agents The second asset, the asteriskAI Indexasterisk, is a real page on your site —unlike llms.txt, which is a technical file— and serves three audiences at once that are often overlooked. It serves your asteriskinternal teamsasterisk, who finally have a single place to consult the official truth on any topic. It serves your asteriskpartnersasterisk, who can refer to a canonical source instead of scattered versions. And it serves asteriskagentsasterisk, who find in it a readable map of your knowledge. This triple service is what makes the AI Index one of those investments that pays off far beyond GEO. A good AI Index should contain your canonical definitions —directly from the Entity Sheet—, your sources of truth organized by topic, your policies and contact information, and a changelog of recent changes. That changelog deserves special mention because it turns the AI Index into a living document: a dated record of what changed and when tells both your team and agents that the page is maintained, and that freshness signal increases trust in everything else. ### Reading paths: from intent to route The third asset links directly to the work from Module 3, but here you formalize it as an explicit guide. A asteriskreading pathasterisk is a recipe that translates a specific intent into the sequence of pages that satisfies it. For example, for the question "how do I get started?", the path would be: the answer-first page is "Getting started," the source of truth is the documentation, and the relevant policy is bot access. Written that way, the path tells the agent exactly where to go and in what order. The value of writing these paths goes beyond guiding the agent: the exercise itself forces your routes to be short and coherent. When you try to write the recipe for an intent and discover you cannot complete it because a page is missing or the path takes too many turns, you have just found an architecture defect to fix. Reading paths are, at once, a guide for AI and an audit of your own structure. ### llms.txt and AI Index, illustrated Let us see how all of this materializes. Here is what a minimal but complete llms.txt would look like, with its versioned header, description, topics, sources of truth, and notes for agents: ``` # llms.txt — BotPass Last updated: 2026-06-03 Version: v1 ## What this site is - BotPass helps creators track AI bot consumption and convert it into value. ## Core topics - GEO basics: https://example.com/geo - AI bot tracking: https://example.com/docs/bot-tracking ## Sources of truth - Pricing: https://example.com/pricing - Bot access policy: https://example.com/policy/bot-access - AI Index: https://example.com/ai-index ## Notes for agents - Prefer canonical URLs above. - If a clean view exists, read it first. ``` Notice how little space it takes and how much it says: at a glance, an agent knows what you are, what your topics are, where your truth lives, and how you prefer to be read. The AI Index, meanwhile, would follow a structure like this: a TL;DR at the top, sources of truth —pricing, policies, docs—, definitions from the Entity Sheet, and a changelog of recent changes. Together, they guide an agent without leaving room for guesswork: llms.txt orients from the domain root and the AI Index gives detailed reference when they enter. 🎓 ## 🎓 Academy Level — llms.txt, AI Index, and reading paths in detail The above is open level. From here: full methodology, templates, practical labs, module checklist, and the asterisk🏁 Certification milestoneasterisk — free when you register at botpass.io. ### 🔒 Academy Level content Register free at botpass.io (or sign in) to unlock the full module, templates, and certification milestone. [Sign in](https://botpass.io/app/login?redirect_to=%2Facademy-modulo-7%2F)[Create free account](https://botpass.io/signup?redirect_to=%2Facademy-modulo-7%2F) ### 🎓asteriskAcademy Level.asterisk The above is open level. Here you unlock a 💡 commented llms.txt ready to adapt, AI Index structure, a publish-and-verify lab, and hacks. Requires a free account at botpass.io. Explained in more detail. ### llms.txt, robots.txt, and sitemap: three cousins that do not compete One of the most frequent confusions is mixing these three files, so it is worth separating them clearly because they do different, complementary things. asteriskrobots.txtasterisk restricts crawling: it tells crawlers what they can and cannot access; it is an access instruction. The asterisksitemapasterisk lists URLs: it exhaustively enumerates your site's pages so search engines can discover them; it is an inventory. And asteriskllms.txtasterisk recommends reading: it points out, from everything accessible, what matters and in what order to read it; it is an editorial guide. They do not compete at all; they complement each other as three layers: one controls the door, another inventories the rooms, and the third tells you which room to start in. A site well prepared for AI has all three, each doing its job. ### What agents actually respect: honest expectations It would be dishonest to sell you llms.txt as an order every agent obeys, and that honesty matters so you calibrate your effort correctly. The reality, today, is that llms.txt is a young proposal and adoption is uneven: some systems take it into account, others do not yet. So the right way to think about it is as a asterisksignal that tips the scaleasterisk, not a switch that guarantees an outcome. Does that mean it is not worth it? On the contrary: it is cheap to create and maintain, has no downsides, aligns with a clear industry trend, and to the extent it is respected it benefits you. It is an asymmetric bet —low cost, possible growing benefit— and those are exactly the bets worth making. What you must not do is build your entire strategy on the assumption that it is obeyed to the letter; build it on architecture and content, and let llms.txt add on top. ## 📋 Templates asteriskTemplate A — llms.txt v1 (commented).asterisk Adapt this base by replacing the placeholders with your data, and respect the order: description, topics, sources of truth, notes. That order is not accidental; it reflects the priority with which you want an agent to process your site. ``` # llms.txt — [Your brand] Last updated: YYYY-MM-DD Version: v1 ## What this site is - [1-3 lines: what you are and who you serve] ## Core topics - [Hub 1]: https://... - [Hub 2]: https://... ## Sources of truth - Pricing: https://... - Policy: https://... - AI Index: https://... ## Notes for agents - Prefer canonical URLs above. - If a clean view exists, read it first. ``` asteriskTemplate B — AI Index structure:asterisk TL;DR · Sources of truth by topic · Canonical definitions (Entity Sheet) · Policies and contact · Changelog. Keep the changelog current: it is the part easiest to forget and the one that adds the most freshness signal. ## 🧪 Practical labs asteriskLab 1 — Publish your llms.txt v1 (40 min).asterisk Draft it with the template, prioritizing your canonicals, and publish it at your domain root. Generate or validate it from BotPass to ensure the format is correct and URLs resolve. Resist the temptation to lengthen it: if your first version fits on one screen, you are on track. asteriskLab 2 — Create the AI Index and verify (30 min).asterisk Publish the AI Index page with its full structure and, after a few days, return to BotPass to check whether bots start consuming your canonical sources of truth more consistently. That deferred check is what turns the module into a closed improvement loop rather than an act of faith. ### ⚡ Hacks Three reflexes so your reading guide works. asteriskShort and prioritizedasterisk: remember llms.txt is not a full sitemap, it is your three to five hubs and your canonicals, and every extra line weakens the ones that matter. asteriskClean viewasterisk asteriskfirstasterisk: tell agents explicitly to read the clean view when it exists, connecting this module with the previous one. And asteriskversion and dateasterisk: include `Version` and `Last updated` so the agent knows the document is alive and maintained, which increases the chance it will be taken into account. ### 💎 Additional high-value content On asteriskllms.txt vs. robots.txt vs. sitemapasterisk, you already have it covered: internalize that one recommends reading, another restricts crawling, and another lists URLs, and far from competing they complement each other as three layers of AI preparation. On asteriskwhat agents actually respectasterisk: keep realistic expectations, llms.txt guides but does not compel, and that honesty helps you invest the right amount of effort. And on the asteriskAI Index as internal hubasterisk: do not see it only as a GEO asset, but as a single source that also aligns your own team and partners, multiplying the return on the work. ## 📚 References and recommended reading - External sources on llms.txt and reading guidance for agents. They complement Academy Level; they do not replace hands-on practice with BotPass. - [llmstxt.org — The /llms.txt file — the official specification of the proposal. ↗](http://llmstxt.org/) - [Answer.AI — /llms.txt: a proposal to help LLMs use websites — Jeremy Howard's original post with reasoning and examples. ↗](http://answer.ai/) - [Ahrefs — What Is llms.txt, and Should You Care About It? — critical, realistic view on current adoption. ↗](https://ahrefs.com/blog/what-is-llms-txt/) - [Semrush — What Is LLMs.txt & Should You Use It? — when it is worth implementing and how. ↗](https://www.semrush.com/blog/llms-txt/) 1Agent arrivesat your domain2Reads llms.txtpurpose + structure3Reading listprioritized pages4AI Indexpolicies, sources, contactllms.txt as "recommended reading": guides the agent toward what matters.### 🏁 Certification milestone (Module 7) ### 🏁asteriskBotPass milestone (verifiable): llms.txt v1 publishedasterisk (generated/validated 💡 from BotPass) + AI Index page created + follow-up verification in the dashboard: are bots starting to consume the intended canonical sources of truth? Counts toward your asteriskCertified GEOasterisk asteriskPractitionerasterisk certification. 📣 asteriskShare your progress (optional):asterisk"Today I published my site's llms.txt: a 'reading list' for AI agents with my canonical sources of truth. Here is how I structured it (and how I check with BotPass whether bots respect it)… #GEOAcademy" asterisk🏁 Module milestone reached?asteriskWhen you have completed the practice and the milestone is recorded in your BotPass plugin, mark the module. After completing all 10, submission for review unlocks. Mark module 7 as completed [← Previous asterisk🛡 Module 6 · Clean content and access controlasterisk](https://botpass.io/academy-modulo-6/?botpass_lang=en) [Next →asterisk📣 Module 8 · Authority and distribution: where citations are bornasterisk](https://botpass.io/academy-modulo-8/?botpass_lang=en)