Skip to content

v2.31.0 — deleting a capability is not removing it

Latest

Choose a tag to compare

@hamza-ali-shahjahan hamza-ali-shahjahan released this 03 Sep 23:11
· 8 commits to main since this release
edb9883

You removed something for a reason: it cost money, it leaked data, it was slow. You deleted the calls, updated the comments, review passed. Six weeks later it is back — added in good faith by someone who never knew why it left, because nothing stopped them.

A deletion is a state. A check is a property. Only one survives a growing codebase.

What happened

A product in this portfolio removed a metered API from every user-facing path. Every call site deleted. Comments updated. Review passed.

The SDK was still in the module graph — through import type { X } from "the-sdk" in a module the clearance pipeline imports. Erased at compile time, so it could not actually run anything, but sitting one keystroke from a value import in a file reachable from a user request. No diff review catches that.

Three things that look like enforcement and are not

An environment variable. A flag existed, defaulted off, documented. It is a switch a server can flip — and "a server should not be able to do this" was the entire reason for the removal. Delete the variable; don't default it to off.

Deleting the call sites. You will miss one. See above.

Comments. Read only by people who already agree with you.

bun run check-boundaries

Declares invariants in .import-boundaries.json and enforces them on the real import graph:

{ "boundaries": [{
    "name": "no metered SDK from user-facing paths",
    "forbidden": ["@some/metered-sdk"],
    "roots": ["src/app/**/route.ts", "src/lib/tools/**/*.ts"],
    "allow": ["src/app/api/admin/approve/route.ts"],
    "why": "Costs $1.41 a call. Acquisition is approval-gated — see ADR-0006."
}]}

Four things make it hold rather than rot:

  • Globbed roots. A hand-maintained list silently loses coverage as the app grows — the first hand-written version listed 10 entry points for an app that had 39.
  • Exceptions in config. An exception in config is a decision. An exception in someone's head is a hole.
  • A written why. So the next person can tell "you are violating this" from "this boundary genuinely moved."
  • Value vs type-only edges. import type is erased — it cannot load a module or break an install. A dependency boundary that counts it reports impossible failures. An adjacency boundary may want it. Per-boundary, defaulting to the runtime truth.

It found two bugs in its own config in ten minutes

Pointed at this repo's own "zero runtime dependencies" promise:

  1. It followed type-only edges and reported nine impossible failures — the chain to zod runs through import type, so telemetry works fine on a root with no node_modules.
  2. Its roots included dashboard/, a separate Next.js app with its own package.json.

Both fixed, then verified by planting a deliberate violation and watching the check go red. A green check proves nothing until you have seen it fail.

Also in this release

silence-as-a-failure-mode — three bugs in one build that passed 140 assertions, because in each case the channel that would have reported them was muted: a swallowed .catch(), a custom column type with a baked-in name, and a bare coverage/ in .gitignore that silently excluded a real source directory. git add -A said nothing; the local build passed because the file was on disk; only CI's clean checkout could see it. For anything you cannot see directly, assert its effect once — read the row back after writing it.

a-metric-that-cannot-discriminate — a rank-fusion score reads 0.0148 for an excellent match and 0.0115 for a poor one. A whole detection layer was about to be built on a signal carrying no information about the thing being detected. Test any metric against a known-good and a known-bad case before you hang a decision on it.

Calibrating a constant — run the cheapest experiment that separates your classes before hard-coding the number. Six probe queries put in-domain similarity at 0.581–0.662 and genuinely-absent topics at 0.457–0.491, so the threshold went at 0.55: the middle of a measured gap, not a round number that sounded reasonable.

Verified: every CI gate passes, 241 tests green.