Codified is not implemented

Every long-running engineering project develops a small library of hard-won rules. A flag that has to be set in a particular environment, or the system runs at one-hundredth its real speed. A startup ritual that has to happen in a specific order, or three components fight for the same resource and lose. A configuration file whose contents are the result of three weeks of empirical work, where every value is non-obvious and every value matters.

Each of those rules typically gets discovered the same way: the system breaks, somebody figures out why, and the lesson gets written down. The rule lives in a comment, or a README, or a wiki page, or, if you're being a little more careful, a "decisions" document with the date and the reason. The team feels like the lesson is now safe. It has been codified.

Six weeks later, the same rule is broken again. Not always by a different person; sometimes by the same person who wrote the rule down. The break looks accidental from the outside — "we made a small simplification" — but it is actually the predictable failure mode of treating a paragraph as a guarantee.

Why a rule that is written down still drifts

Three reasons keep showing up in the post-mortems. None of them is unusual; all of them are universal in any project that runs longer than a sprint.

Inheritance. A new component reuses an old one without re-reading the rules the old one was designed under. A new endpoint copies a config from an adjacent endpoint. A new build script borrows a fragment from an older one. The borrowed code carries assumptions the new context doesn't honour, and the rule that was true in the original context silently does not hold in the new one. Nobody is wrong; the rule is just not where the new author was looking.

Simplification. A future engineer looks at the configuration and thinks, "this looks needlessly complicated." They simplify. The complication, of course, was not needless. It was the precise shape required to satisfy the rule. The simplification removes it. Tests pass. Things run. Nobody notices the regression for weeks, because the regression is not a crash — it's a degradation in a property that nobody is checking.

Forgetting. Six weeks is enough time for a person to forget a rule they themselves wrote. A year is enough time for a team to lose half of them. The rule does not move; the people change, or the people drift, or the people simply stop reading the doc that holds the rule. The doc is still there. The rule is no longer being enforced.

/ The frame

The doc says what should be true. Nothing in the system checks that it still is. Over a long enough timeline, that gap is a leak. Every rule held in a paragraph is a rule waiting to be silently broken.

The fix is not better docs. The fix is a mechanical guard.

Once we accepted that "write it down" doesn't hold over time, we started building every operational rule with two artefacts. The first is the rule itself, in prose, in the project's decisions log, with the date and the reason it exists. The second is a piece of code — a script, a check, a sentinel, a pre-flight — whose only job is to fail loudly if the rule has been violated. The doc explains. The code enforces.

The mechanical guards we keep building have a few common shapes.

The doc says what should be true; the script enforces that it is true. There is no other reliable way to keep a long-running build honest.

Where this has saved us

A few concrete cases, abstracted from the projects they happened on.

A driver flag set in our systemd unit. Without this flag, the inference workload runs on the wrong code path and produces nonsense output that, infuriatingly, looks shape-correct — the right number of dimensions, the right ranges, the right shape. Logs say "200 OK." The downstream system silently breaks. We codified the flag. Three sessions later, somebody wrote a new unit file and didn't include the flag. The pre-flight script — which inspects the running unit's environment, not the file on disk — refused to start the job. The doc was already there; the doc would not have caught this. The script did.

A startup ritual that has to be sequential. A particular class of resource, when initialised in parallel, fights itself. The codified rule was "initialise these one at a time." Six weeks later, an enthusiastic optimisation pass turned the loop into a parallel one because "the sequential version was slow on startup." The break was silent: the system came up looking healthy and ran at a fraction of its expected throughput because half the resources were in a degraded state. Now the initialisation function reads from a configuration profile with a hard "sequential = true" entry, and a check verifies the profile against an expected hash before launch. The optimisation cannot be reintroduced without explicitly editing the profile, which requires explicitly disagreeing with the codified rule.

A configuration that pinpoints a specific resource composition. A pool of similar-but-not-identical resources, where a particular subset is required for performance reasons that are not obvious from looking at any single resource. The codified rule named the subset. Two sessions silently regressed it because a minor cleanup pass "looked tidier" with a different default. Now a pre-flight script asserts the running composition against the codified one and refuses to start if they disagree. The cleanup is fine; it just has to declare itself.

The discipline this asks for

Building a mechanical guard for every codified rule is more work than writing the rule down. Not by a lot — a typical guard is twenty to fifty lines of code — but it is an extra step at the moment of greatest temptation, because the moment you have just discovered a rule is also the moment you most want to declare victory and move on. The discipline is to take the extra hour and write the script.

The compounding return is large. Six months in, the rules you codified with guards are still being honoured; the rules you codified in prose are quietly drifting. The difference is invisible until something breaks, at which point the difference is the only thing that matters.

None of this is AI-specific. It applies to any long-running engineering work where lessons accumulate and people change. A backup procedure that is documented but never tested is a procedure that does not work. A code style that is in a style guide but not enforced by a linter is a style that diverges. A dependency-version constraint in a doc but not in a lockfile is a constraint that drifts. The shape is the same: prose alone is not a guarantee. A check that fails loudly on violation is.

---

If you are running a long-lived engineering build — or about to inherit one — and you suspect the operational rules are drifting, that is a conversation our 30-min calls are for. Book one, or write to help@digicrafter.ai.

---