Gitignored Build Caches Must Ship to Vercel

A cache that is gitignored does not exist on Vercel. On an on-demand route that means live fetches inside the request and a write to a read-only disk.

Path
reminders/Gitignored-Build-Caches-Must-Ship-To-Vercel.md
Authors
Michael Staton
Augmented with
Claude Code on Claude Opus 5 (1M context)
Tags
Reminder · Tree-Wide · Vercel · Astro · Deployment · Build-Caches · Verification

Gitignored Build Caches Must Ship to Vercel

Rule. Any app deployed to Vercel — Astro, Svelte, or otherwise — that reads a build-time cache at runtime must ship that cache with the deploy. Track it in git and add it to the adapter’s includeFiles. A cache in .gitignore does not exist on Vercel, and the route that depends on it will fail there while working perfectly on your machine.

Known instances in this tree:

Cache Consumer Written by
src/data/og-cache.json LFM Open Graph link previews lfmOgFetcher
src/data/youtube-playlist-cache.json playlist surfaces build-time fetcher
.essays-cache/ essays fetcher build-time fetcher

What actually happened (2026-09-25, mpstaton-site)

A gated proposal route returned HTTP 500 in production and rendered fine in local dev. src/data/og-cache.json was gitignored, so it never reached Vercel. The gated routes render on demand, so with no cache every external link in the proposal — 147 of them — became a live HTTP fetch inside the request, and then the dispatcher tried to write the results back to a read-only serverless filesystem. Either half alone is a 500.

It rendered fine locally because the machine has the cache and a writable disk.

Why the assumption breaks

The OG stage was written for build time, where two things are true that stop being true on an on-demand route:

  1. A cache miss is cheap. In a build it is. In a request it is a live HTTP fetch per link, and a long document has hundreds.
  2. The cache file is writable. On a serverless host the bundle is read-only, so the dispatcher’s flush() throws EROFS the moment anything goes dirty.

Every /promote/* and /proposals/* route in mpstaton-site sets prerender = false. That single line is what moves the cache from a build concern to a request-path dependency.

The five things to get right

  1. Track the cache in git. Un-ignore it, with a comment saying why, so the next person does not “tidy” it back into .gitignore.
  2. Add it to includeFiles. Tracking it in git is not enough — the Vercel adapter bundles only what it traces or what you name explicitly. In mpstaton-site that is listContentFiles() in astro.config.mjs. Globs fail with ENOENT; expand to literal paths.
  3. Resolve the cache path absolute. A relative path resolves against process.cwd(), which is a different directory in a dev server, a local build, and a Vercel function root. A relative path let one render read one file and write another.
  4. Send writes to /tmp on serverless. It is the one writable location a function gets. Read from the bundled copy, write to a /tmp clone.
  5. Make a miss survivable and fast. Long TTLs so the tracked cache cannot lapse back into request-time fetching; no retries and a short timeout on serverless; and a try/catch that falls back to a plain parse. Enrichment is an enhancement — it should never be able to take the page down.

If the cache did not ship at all, skip enrichment outright rather than attempting it. There are no previews to recover either way, and trying only spends the request on fetches that cannot be saved anywhere.

How to verify before pushing

Local pnpm dev success is not evidence. Run the built function the way Vercel runs it:

pnpm build
cd .vercel/output/functions/_render.func
chmod -R a-w src/data            # simulate the read-only bundle
VERCEL=1 node /path/to/probe.mjs # import dist/server/entry.mjs, call app.fetch()
chmod -R u+w src/data

The entry’s default export is a fetch-style handler ({ fetch }), so a probe can call it with a plain Request — no HTTP server needed. Run it three ways: cache present, cache missing, and /tmp clone cleared. All three must return 200.

Also worth knowing: Vercel’s bot protection 403s bare curl POSTs. A 403 on an unlock or form endpoint from the command line is the WAF, not your gate — send a browser User-Agent, Origin, and Referer before concluding anything.

The general lesson

A deploy’s inputs are filtered by .gitignore; your local run’s inputs are not. Whenever something works locally and 500s in production, the first question is not “what code differs” but “what files differ” — and after that, “what is writable.” Both were the answer here.

See also: [[Check-The-Substrate-Before-Reasoning-On-Top-Of-It]] — same family of error, one layer down.

References

  • astro-knots/sites/mpstaton-site/src/lib/promote/og-fetch.ts — the shared config that encodes items 3–5 above
  • mpstaton-site commit cfeadf9 — the fix, with the full diagnosis in the body