How RenderCache Is Tested

RenderCache makes a few promises that matter more than any feature. A crawler never waits for Chrome. A failed refresh never throws away a good page. A page with a cookie never ends up in a shared snapshot. Each promise has a test that tries to break it, and those tests run on every change, on Linux, macOS and Windows, with a real Chrome.

The figures below come from the run behind 0.2.0. They’re copied from the recorded results in the repository, which a script writes from the CI run, so nobody types them in.

Killing the process mid-save

A snapshot is written to a temporary file and then renamed over the old one, so a reader sees the old page or the new one and nothing in between. The test checks this the blunt way. It starts a child process that keeps saving new versions of one page, kills it at a random moment, then opens the snapshot. It does this 60 times on each system.

The kill timing is random, so how many land mid-write changes from run to run. None of them left a damaged file.

The origin goes down

With a saved page past its expiry date, the test shuts the origin down and makes every refresh fail. Then it sends 50 crawler requests. All 50 got the saved copy on every system, the failed refreshes retried in the background, and the saved copy was still there afterwards. People asking for the same page saw the origin’s error, as they should: the saved copy is for crawlers.

Chrome gets stuck

This test makes the renderer hang and never return, then sends 100 crawler requests, half for a saved page and half for one that was never rendered. The saved page comes from disk. The new one goes straight to the origin and is queued for rendering later. No request waits for Chrome.

These are test machines on shared CI runners, so the times say nothing about your server. What matters is that none of them is anywhere near a render.

A real page in a real browser

The Chrome test loads a page that builds itself with JavaScript and tries to reach other servers along the way. RenderCache has to capture the finished page and block everything that isn’t the origin.

Most of that time is Chrome starting from cold on a CI runner, so it isn’t a measure of how long your pages take.

The rest of the list

The other promises have their own tests, which pass on all three systems:

• Concurrent refreshes and reads never see a partial snapshot.
• A stale page is served at once and refreshed only once, however many crawlers ask.
• A failed refresh keeps the old copy, and a page that turns private is withdrawn.
• Requests with cookies or credentials never get a shared snapshot.
• Sitemap pages are rendered before the first crawler arrives and survive a restart.
• Files in the data directory are never opened through a link.
• Only one process can own a data directory.
• Only a trusted edge can set the forwarding headers the origin sees.

In all, 85 tests passed on each system, with the race detector on. The release itself is checked again: each archive is built twice and must produce the same checksum, then it’s extracted and run against Chrome on its own system before anything is published.

What this doesn’t show

Tests can only check what someone thought to test. They show that RenderCache keeps its promises on GitHub’s Linux, macOS and Windows runners, not on your hardware, behind your edge, with your app. The deployment notes list what still needs checking on your own host, such as whether Chrome’s sandbox works there. And nothing here can show that a crawler that got a page went on to index it or cite it.

Download 0.2.0, or read the tests themselves.