Agentic Engine Optimization: Google's AEO Framework

What Osmani Actually Published

The April 11 post frames a specific problem. AI coding agents — Claude Code, Cursor, Cline, Aider, Codex CLI, Gemini CLI — consume documentation fundamentally differently from humans. They issue a single HTTP request, strip HTML, count tokens, and either use your content as context or silently discard it. Human-optimized documentation patterns (progressive disclosure, accordion sections, tabbed code samples, "click to expand") actively hurt agent consumption. Osmani's framing: "agents want everything at once."

The framework has four concrete artifacts:

• A five-signal modelfor evaluating any site's agent-readiness
• A six-layer stack in implementation order — what to ship first, second, third
• A tiered token budget for documentation pages by purpose
• The agentic-seo open-source CLI that scores a 100-point audit against these criteria and outputs JSON for CI/CD

Agencies should treat this the same way the industry treated Core Web Vitals when Google announced them in May 2020: a specific, measurable set of technical criteria that suddenly becomes the shared vocabulary for a capability everyone had been discussing in vague terms. The parallel is imperfect — Google will not rank on AEO scores directly — but the industry value of a shared audit is the same.

The Five AEO Signals

Osmani's framework organizes everything around five signals that any agent-accessible site should pass:

The five signals map to a six-layer implementation stack, in recommended shipping order:

1. Access control — robots.txt rules that explicitly allow or deny AI crawlers
2. Discovery — llms.txt at the domain root as an agent-readable sitemap
3. Capability signaling — skill.md files describing what each service, API, or feature does
4. Content formatting — semantic HTML and Markdown twins of every page
5. Token surfacing — per-page token counts published as metadata
6. Copy-for-AI UX — copy-to-clipboard, raw-view links, copy-for-AI buttons

The Tiered Token Budget Rule

Popular X posts summarized Osmani's token guidance as "keep docs under 15k tokens." That is the quick-start number only. The real framework is tiered by page purpose:

The additional rule: front-load answers within the first 500 tokens. Agents parse linearly and stop early once they have an answer. Buried answers cost the entire page's tokens without delivering the expected signal.

Why <15k Beats a 2M Context Window

Token budgets look small compared to April 2026 frontier context windows. Here is the current landscape:

Against a 2M context window, a 30,000-token documentation page is 1.5% of total capacity. So why does Osmani's budget still matter? Three reasons:

• Cost economics. GPT-5.4 charges a 2× input premium above its 272k standard window. Claude Opus 4.7 charges a premium above 200k. A single oversized page can flip an agent workflow from cheap to expensive inference.
• RAG-chunk efficiency.Most production agents don't dump whole pages into context — they chunk, retrieve, and compose. A 30k-token page chunked into 6 × 5k retrievable units preserves the useful parts without carrying the whole page's overhead.
• Agent branching. A Claude Code session might consume documentation pages from 5–10 different services in parallel. Even at 1M capacity, ten 30k-token pages burn 30% of the window before the agent has written a single line of code.

Small budgets aren't about fitting into today's windows. They're about the per-interaction economics and the composition pattern real agents use.

llms.txt: Adoption Status in April 2026

llms.txt was proposed by Jeremy Howard (Answer.AI) on September 3, 2024 at llmstxt.org. The format is Markdown: H1 site name, blockquote summary, optional paragraphs, H2-delimited link sections, and an "Optional" section for skippable content. Nineteen months later, here is the verified adoption status:

The honest read: llms.txt adoption is meaningful for agent-consumption workflows (Claude Code pulling documentation, Perplexity researching a topic, ChatGPT Advanced Research fetching sources). It is not yet a Google Search ranking factor. Those are two separate value propositions — both real, different audiences.

AGENTS.md: Four Vendors, One Emerging Spec

AGENTS.md is now stewarded by the Linux Foundation's Agentic AI Foundation at agents.md. The format is plain Markdown — no required sections — because, in the spec's framing, "the consumer is a model trained on markdown, and the file's primary job is to be read, not to be parsed." Vendor support as of April 2026:

The agency-facing recommendation: publish an AGENTS.md as the primary file at repo root, and add a short CLAUDE.mdthat either mirrors or references it for teams specifically using Claude Code. Common sections that actually help agents (none are required by the spec): project overview, build and test commands, style conventions, architecture notes, and — critically — "things the agent should never do."

Markdown-First: HTML Token Cost Data

Osmani's insistence on Markdown twins has a measurable justification: for the same content, HTML consumes meaningfully more tokens than Markdown because of wrapper divs, CSS classes, ARIA attributes, inline styles, and rendered navigation chrome. The magnitude depends on page type. Here are the reported ranges from third-party measurements:

A safe agency-side operating assumption: ~30% token savings on typical content pages, up to 90% on app-shell-heavy pages. For large documentation sites, that directly translates into pages fitting under Osmani's budgets without content cuts. For marketing sites, it means a single Markdown twin delivers content agents can actually use without forcing them to parse your nav, footer, cookie banner, and pop-up modal six times.

Fusing AEO With the Adobe 42% Data

On April 16, 2026, Adobe Digital Insights released its April 2026 retail report covering more than 1 trillion U.S. retail visits plus a 5,000-consumer survey. The headline: AI-referred traffic now converts 42% better than non-AI traffic. Twelve months earlier, the same data set showed AI traffic converting 38% worse than non-AI. That is an 80-point swing.

Vivek Pandya, Director, Adobe Digital Insights: "Notably, AI traffic continues to convert better (visits that result in purchases) than non-AI traffic, which covers channels such as paid search and email marketing."

Fusing Adobe's numbers with Osmani's framework gives agencies an ROI equation for AEO adoption. If a site captures X% of sessions from AI sources, and those sessions convert at 1.42× the baseline rate, improving AEO compliance directly lifts the top of that funnel. Modelled across realistic agency client scenarios:

The math assumes the AI-traffic conversion lift holds at the new blended share — a plausible but not guaranteed assumption. The point isn't to promise exact revenue. The point is that AEO work now has a direct revenue hook instead of a vague "be AI-friendly" pitch, which changes how agencies position the deliverable in client proposals.

Auditing a Client Site Against the Framework

Osmani's open-source agentic-seo CLI scores 100 points across five categories. Install and run:

# One-off audit of a live URL
npx agentic-seo --url https://client.example.com

# CI/CD: fail the build below 80 / 100
npx agentic-seo --url https://client.example.com --threshold 80 --json

# Local build-output audit (Next.js, Docusaurus, Astro, Hugo, Jekyll, etc.)
npx agentic-seo --serve

The 100-point scoring breakdown:

The tool auto-detects Next.js, Docusaurus, Eleventy, Astro, Hugo, Jekyll, Gatsby, VitePress, MkDocs, and Sphinx, and can emit JSON (--json) for wiring into GitHub Actions or Vercel build steps. Osmani's README explicitly calls out that the project is not endorsed by Google — it's an independent open-source effort the framework author happens to lead.

The 30-Day Agency Rollout Plan

For agencies integrating AEO into client engagements, a 30-day rollout keeps scope tight and produces measurable deliverables:

The week-4 before/after score report is the delivery artifact that justifies the engagement. Agencies should budget 20–40 hours per client site depending on size — discovery is cheap, markdown twins are the heavy lift.

Conclusion

AEO is what GEO needed: a specific, auditable, open-source methodology from an authority source, published with a working implementation tool. Google itself will not rank on AEO scores — John Mueller's public comments are clear on that — but agents increasingly depend on the signals Osmani enumerates, and Adobe's April 2026 retail data shows the commercial stakes are real and rising.

Agencies that ship the five-signal audit, publish the baseline artifacts (robots.txt, llms.txt, AGENTS.md, Markdown twins), and tie the work to measurable AI-traffic outcomes will own the next 12 months of client conversations. The ones that keep pitching "AI SEO" as a vague concept will lose those conversations.

Osmani's closing line on the April 11 post: "the teams that move early here will probably have a real advantage: their APIs will be the ones agents recommend." The same logic applies to marketing sites, documentation, and product pages. Early movers get cited; late movers get skipped.

Frequently Asked Questions