Technical SEO and rendering, defined.

AI Overviews

Google's AI-generated answer block that appears above traditional search results.

Answer Engine Optimization (AEO)

Optimizing content to be cited inside AI-generated answers.

Canonical tag

An HTML hint that tells search engines the preferred URL for a piece of content.

ClaudeBot

Anthropic's crawler. Used for training and Claude product context.

Cloaking

Serving materially different content to crawlers vs users — explicitly forbidden by Google.

Core Web Vitals

Google's headline page-experience metrics: LCP, INP, CLS.

Crawl budget

The number of URLs a crawler is willing to fetch from your site within a given window.

Crawled, currently not indexed

A Search Console state: Google fetched the URL but decided not to add it to the index.

Cumulative Layout Shift (CLS)

How much visible content unexpectedly shifts during a page lifetime.

Discovered, currently not crawled

A Search Console state: Google found the URL but has not yet fetched it.

Entity SEO

Optimizing for clear entity identification in search and answer engines.

GPTBot

OpenAI's training crawler. Distinct from OAI-SearchBot.

hreflang

A signal that tells search engines which language and region version of a page to serve.

Hydration

The process of attaching JavaScript event listeners to server-rendered HTML in the browser.

Incremental Static Regeneration (ISR)

A hybrid where pages are static-served from cache but regenerated on demand or on a TTL.

Interaction to Next Paint (INP)

Worst-case latency between a user input and the next visual update.

JSON-LD

JSON-formatted structured data that Google and AI engines extract for entity understanding.

Largest Contentful Paint (LCP)

The render time of the largest content element visible in the viewport.

llms.txt

A proposed file at site root that hints to AI crawlers what content to prioritize.

OAI-SearchBot

OpenAI's search crawler. Indexes content for ChatGPT search-time grounding.

PerplexityBot

Perplexity's crawler. Indexes content for Perplexity AI answer citations.

Prerendering

Generating static HTML snapshots of JavaScript-rendered routes for crawlers and (sometimes) users.

RFC 9110 (HTTP Semantics)

The current IETF standard defining HTTP semantics: methods, status codes, headers.

RFC 9309 (Robots Exclusion Protocol)

The IETF standard that formally specifies robots.txt parsing and behavior.

robots.txt

A file at site root that tells crawlers which paths they may or may not fetch.

Server-Side Rendering (SSR)

Generating HTML on the server for every request, then sending it to the browser.

Soft 404

A page that returns HTTP 200 but appears empty or 404-like to crawlers.

Static Site Generation (SSG)

Building every page as static HTML at build time and serving from a CDN.

Time to First Byte (TTFB)

The time between request and the first byte of response from the origin or CDN.

Time to Interactive (TTI)

The time until the main thread is reliably free to handle user input.

XML sitemap

An XML file listing the URLs you want crawlers to discover and prioritize.