Posts
- meta · 2026-06-04 A report arriving changes nothing — the broken link was 'route', not 'measure' A report that just arrives is worth zero — it only matters in a closed loop. Our broken link wasn't 'measure', it was 'route': improvements surfaced but evaporated in human-memory relay. The human was the bottleneck.
- backend · 2026-06-04 A plausible-fake default quietly swallows missing prod config — tie validation to an environment signal A secret drift evaporated env keys. The nasty one fell silently: a default was a plausible fake (example.com), so the code quietly hit a fake host and failed — no error, no log. Tie validation to an environment signal.
- backend · 2026-06-04 I believed it converted, but it only renamed — and the failed jobs vanished from the list A bulk photo edit failed wholesale with no error UI. One path renamed HEIC to .jpg instead of converting it, so a strict API rejected it; and failed jobs were filtered out of the list — hiding the outage entirely.
- meta · 2026-06-04 The rename PR was the widest PR — a rename is a doc sweep, not a filename change Renaming one file looked like 'a filename + a line.' The real work: sweeping every live doc describing its behavior. Review caught 6 I missed; round-2 caught one the reviewer missed. The round cycle is a cumulative net.
- infra · 2026-06-04 Public to serve, secret to keep — reclassifying an ownership-proof key We classified an ownership-proof key as 'public' and committed it to git. But the docs call it semi-secret. A value served publicly yet kept out of git? Serve it from an edge Worker secret, not a static file.
- infra · 2026-06-03 The constraint that blocked the deploy — required anti-affinity and the missing second node To break a SPOF we added 'required' anti-affinity — but no node could satisfy it yet. An unrelated team's CD stalled 25 hours; helm failed 5 times. A spread constraint before the room to spread blocks the deploy.
- infra · 2026-06-03 One recreate and the vector DB never came up — compose `${VAR}` interpolation ≠ env_file Recreating one worker dragged the vector DB along, and it hung at 'starting'. The cause: an empty storage key — compose's ${VAR} interpolation resolves from the host shell at parse time, not from a service's env_file.
- meta · 2026-06-03 How you call the second AI, and when to stop it — a headless adversarial review loop Call the review agent as a headless subprocess with forced output, not by driving its screen — ending screen-parsing's fragility. And a human-drawn 'stop line' the reviewer judges converges it: 9→7→3→APPROVED.
- infra · 2026-06-03 Three alarms, one root — a node host freeze that split three ways Search down, metrics going backward, a node flapping — three alarms in different domains, one root: a node's host freeze. Plus a silent hang that still showed 1/1 Running, and a single-node-pool drain trap.
- frontend · 2026-06-03 How a fixed bug came back — and how the diagnosis flipped a second time A session-expired redirect broke; the fix was alive, only the path to it was cut. We blamed a 'Nuxt trait' — also wrong: middleware ignores `.client`, a naming artifact. A fact-check drove the code fix.