Skip to content

Integration guide

Prerendering on Cloudflare

Cloudflare Workers run before the request reaches the origin, making them ideal for bot detection and prerender routing. The pattern works for any origin (Vercel, AWS, self-hosted) — the Worker decides where to send each request based on user-agent and verified-bot signals.

Setup overview

Deploy a Worker that intercepts all requests, classifies the user-agent, and rewrites bot requests to a prerender service. Use Cloudflare's Bot Management or IP Lists for reliable verification.

Working snippet

// worker.js
const PRERENDER_TOKEN = 'your-token'
const PRERENDER_HOST = 'service.prerender.io'

const BOT_UA = /Googlebot|OAI-SearchBot|PerplexityBot|ClaudeBot|GPTBot|Bingbot/i

export default {
  async fetch(request, env) {
    const ua = request.headers.get('user-agent') ?? ''
    if (!BOT_UA.test(ua)) return fetch(request)

    const url = new URL(request.url)
    const target = new URL(`https://${PRERENDER_HOST}/${url.toString()}`)
    return fetch(target, {
      headers: {
        'X-Prerender-Token': PRERENDER_TOKEN,
        'User-Agent': ua,
      },
      cf: { cacheTtl: 3600, cacheEverything: true },
    })
  },
}

Decision criteria

Fits when

  • Cloudflare is already the CDN or proxy in front of the site.
  • Origin is on AWS, Render, Fly, or any non-Vercel platform.
  • You want bot management (Bot Score, IP Lists) for verification.

Decision criteria

Avoid when

  • Site origin is on Vercel and Edge Middleware is sufficient (avoid double-proxy).
  • You cannot afford the additional Workers request budget at your traffic volume.

Common pitfalls

Watch for these on first rollout

Worker timeouts on cold prerender

Workers have a strict CPU and wall-clock budget. If the prerender source is cold (first request after deploy, or cache miss), it may exceed the timeout. Use cf.cacheTtl and warm the prerender cache for top routes.

Conflict with Cloudflare Cache Rules

If Cache Rules cache the SPA HTML, bot requests may hit the cached SPA before the Worker can rewrite. Place the Worker before the cache layer or use Cache Rules bypass for crawler user-agents.

Bot Score not enabled by default

Cloudflare Bot Management gives reliable verification but requires an Enterprise or Pro plan. Without it, fall back to user-agent + IP allowlist verification, which is good enough for major crawlers.

FAQ

Platform-specific questions

How much does a Cloudflare Worker request cost for prerender routing?

Workers Free includes 100k requests/day. Paid Workers start at $5/month for 10M requests. For bot routing, you only invoke the Worker on bot traffic, so cost stays low even for high-traffic sites.

What is the CPU/duration limit per Worker invocation?

Workers Free: 10 ms CPU. Workers Paid: 50 ms CPU. Bot routing logic uses < 1 ms typically. The actual prerender fetch (origin call) does not count against CPU time — only against wall-clock duration (30 s limit).

Do I need Cloudflare Bot Management?

Helpful but not required. For major crawlers (Googlebot, Bingbot, GPTBot, ClaudeBot, PerplexityBot), user-agent + IP allowlist verification is reliable enough. Bot Management is worth it if you need to filter sophisticated scrapers from prerender abuse.

Last updated:

Platform documentation

Authoritative references

Related deep-dives

Want this scoped to your stack?

Generic snippets are useful for orientation; the actual integration needs your routes, your bot traffic mix, your cache strategy. The scoping call is 30 minutes — bring the platform constraints.

Book scoping callBrowse all integrations

Other platforms