An agent-ready website: llms.txt, JSON-LD and WebMCP

Published
Updated

Making a website readable for AI agents is mostly an old discipline of honest HTTP, semantic markup and structured data, not a product to buy. Only two small conventions are new, llms.txt and WebMCP. The work takes a few days of engineering, and the first fix is often a plain correctness bug that has nothing to do with AI.

Our own case: PageSpeed Insights (Lighthouse 13.6) scored our Agentic Browsing category 2 of 3. The failing audit was llms.txt, and the cause sat in the routing, not in missing "AI optimisation".

What the Agentic Browsing category checks

Chrome's documentation describes the category as deterministic audits of how well a site is built for machine interaction, explicitly experimental and based on proposed standards. Testing it requires Chrome 150 or later, and the WebMCP audits also require registering for the WebMCP origin trial.

Four areas are covered: a subset of accessibility audits (names and labels, tree integrity, visibility), because an agent reads the page through the accessibility tree rather than pixels; layout stability measured by CLS; the llms.txt file; and WebMCP tool registration, declarative in HTML and imperative in JavaScript.

Scoring differs from the other categories. There is no weighted 0–100 score, only a ratio of checks passed. The audit for forms without WebMCP attributes is informational and raises no warnings. The ARD audit looks for a /.well-known/ard.json manifest (or the deprecated ai-catalog.json) that catalogues MCP servers, A2A agents and agent skills. Without a manifest the result is "not applicable", so a company site that exposes no agents does not need one.

Status codes are the first agent interface

The audit reported that our llms.txt lacked a required H1 and appeared to contain no links. The problem was that the file did not exist. The i18n routing treated the first path segment as a locale, and the middleware skipped any path containing a dot. So /llms.txt, like any /anything.txt, rendered the home page with HTTP 200 and a canonical URL pointing at itself. The auditor received HTML instead of Markdown.

That is a soft 404. In the Search Console page indexing documentation, Google recommends returning a 404 status for pages that genuinely do not exist. Per the llms.txt audit page, a missing file makes the audit "not applicable", since the file is optional for now. An honest 404 would not have lowered our result. A 200 serving the wrong content failed it.

The fix was a few lines: the layout checks whether the first segment is a supported locale and calls notFound() otherwise. The broader lesson: agents, crawlers and validators read the status first, and a server that answers 200 to everything misleads all of them. Before adding new files, request a few addresses that do not exist, including some ending in .txt.

A good llms.txt is a short map, not a sitemap

The llms.txt specification is a proposal by Jeremy Howard, first published in September 2024 and revised since. The only required element is an H1 with the site's name. After it come a blockquote with a short summary, optional paragraphs, and H2 sections holding lists of links in the form [name](url): notes. A section called "Optional" collects links a reader can skip when it needs a shorter context.

We generate ours from the CMS, so it cannot drift from the site: services, projects, articles, contact and an Optional section, with one file per language. The skeleton:

1# CloudTech Consulting
2

3> Warsaw software house: web applications, system integrations, AWS and Google Cloud.
4

5## Services
6

7- [Service name](https://example.com/en/offer): one sentence from the CMS
8

9## Optional
10

11- [Polish version](https://example.com/llms.txt): this file in Polish

The specification is clear about what the file is not. A sitemap lists every indexable page; llms.txt is a curated overview that fits in a model's context. Robots.txt states what access is allowed; llms.txt helps an agent at the moment it answers.

Nor is it a ranking factor. Google's guide on optimising for generative AI features in Search says Google Search does not use these files, and that keeping one neither helps nor harms visibility. We know of no first-party list of other AI systems that read it, so we treat it as a cheap addition for tools that support the format, not a traffic channel.

Structured data is the durable investment

Every page on our site carries JSON-LD. Organization and WebSite appear site-wide, articles add BlogPosting and BreadcrumbList, and service pages add Service and FAQPage. The organisation has one stable @id across both languages, and page graphs reference it, which tells an agent that both language versions describe the same company.

Google recommends Organization markup on the home or about page, with no required properties; name, address, telephone, logo and sameAs are among those recommended. An article may be Article, NewsArticle or BlogPosting, with author, headline, image and dates recommended.

Know the limits. The same Google guide says structured data is not required for generative AI search and no special markup exists for it, though it is still worth keeping for SEO. According to the Search Central documentation updates, FAQ rich results stopped appearing in Google Search from 7 May 2026. We keep FAQPage for machine understanding of the page and expect nothing from it in the results page.

In the same layer, translated articles point at each other with hreflang, and Google ignores annotations that are not reciprocal. Our sitemap no longer submits noindex pages or invented lastmod dates: Google uses lastmod only when it is consistently and verifiably accurate.

Forms as tools: declarative WebMCP

The declarative WebMCP API turns an ordinary form into a tool an in-browser agent can call. The toolname and tooldescription attributes on the form element register the tool. toolparamdescription on a field describes the parameter; without it, the browser falls back to the label text. It ships through an origin trial, and Chrome says WebMCP is under active discussion and subject to change.

What matters is who submits. Without toolautosubmit, the browser fills the fields and focuses the form, and a person has to press submit. With it, the agent's call submits the form straight away. Our contact form deliberately omits it:

1<form toolname="send_contact_enquiry"
2      tooldescription="Sends a project enquiry to CloudTech Consulting...">
3  <input name="company-website" tabindex="-1" autocomplete="off"
4         toolparamdescription="Spam trap for bots. Always leave this field empty.">
5</form>

An enquiry goes out in a specific person's name, so that person should see it first. The second detail is subtle. An agent sees every field in the form as a tool parameter, including the hidden honeypot meant for bots. Without the toolparamdescription above it might fill that field in, and the spam filter would reject a genuine enquiry.

Order of work and realistic effort

Estimates from our own work on a CMS-driven site, not a standard:

  1. Status codes: 404 for paths that do not exist, including dotted ones, and no redirects to the home page. A few hours to a day.
  2. Sitemap, canonical and hreflang: no noindex pages, real dates, reciprocal language pairs. About a day.
  3. JSON-LD: Organization and WebSite globally, then page types, validated on several templates. One to two days.
  4. Accessibility as an agent sees it: field labels, button names, a stable layout. Usually a day, depending on the starting point.
  5. llms.txt generated from the CMS, one file per language. A few hours.
  6. WebMCP attributes on one or two forms, tested in Chrome with the origin trial. A few hours.

To go through this list against your own site, our contact form already carries these attributes.

What not to do

Do not stuff keywords into llms.txt. Give each link one factual sentence.

Do not publish markup that contradicts the visible page. Google's general structured data guidelines require markup to be a true representation of the page and forbid marking up content readers cannot see. An FAQ that exists only in JSON-LD is a defect, not an optimisation.

Do not add toolautosubmit to forms that send, order or change something on a user's behalf.

Do not buy an "AI SEO" plugin before checking your status codes, and do not make the category score a target. Our post-fix score has not been measured in production yet, and the category may change with the next Lighthouse release.

Sources