RenderCache Alpha 2 Fixes a Crawler Redirect Loop Behind HTTPS Edges

Alpha 1 had a bug you’d only hit in a real deployment. RenderCache sits behind nginx or a Cloudflare Worker, and the edge does the TLS. When a crawler asked for a page that wasn’t cached yet, RenderCache passed the request on to your app. On the way it threw away the edge’s X-Forwarded headers and wrote new ones from its own connection. That connection is plain HTTP from the edge’s address.

So your app was told the crawler came in over HTTP. Plenty of apps answer that with a redirect to HTTPS. The crawler follows it, lands on RenderCache again, and gets the same redirect. A loop, on exactly the pages that weren’t cached yet.

Alpha 2 keeps the visitor’s address, scheme and host when the request comes from an edge RenderCache trusts. That means an address listed in the new trusted_proxies setting (loopback by default, which covers the nginx example), or a request carrying the configured edge token, which is how the Worker proves itself. Anyone else still can’t set those headers. The Worker example now sends the visitor’s address, scheme and host. If your nginx reaches RenderCache from another machine or through Docker’s published port, add that range to trusted_proxies.

Smaller fixes in the same release

Google-InspectionTool is in the default crawler list now. That’s the agent behind Search Console’s URL Inspection and Rich Results Test, which is usually where you go to check a setup works. Before, it saw the empty app. GoogleOther, Amazonbot, YandexBot, Baiduspider and CCBot were added too, and the nginx and Worker examples match.

Pages crawlers find outside the sitemap use a separate allowance, 256 by default. In alpha 1 that filled up and stayed full until a restart. Now a new URL takes the slot of the least recently requested idle page. Its snapshot stays on disk and is still served; it just stops refreshing until someone asks for it again.

Tracking links don’t eat that allowance any more either. utm_* parameters, gclid, fbclid and msclkid are left out of the snapshot identity, so a link with a campaign tag gets the same snapshot as the plain URL. The list is in ignore_query_params. The fixed /api bypass became bypass_paths, for apps that also serve /graphql or /wp-json. And assets no longer carry a Vary header meant for pages, so a CDN in front can cache them normally.

Existing config files keep working, since all the new settings have defaults. Download alpha 2 or read the release notes.