Release Integrity: When Our Extension Shipped Without Its Own Fixes

What happened

This is a true story from developing the browser extension Clean Copy. The project has one core file with all the logic (clean_copy_core.js) and six surfaces that use it: Chrome extension, Firefox extension, CLI tool, bookmarklet, Obsidian plugin and desktop app.

During a review we discovered something uncomfortable: the Chrome extension's background script did not contain the fixes from the last two releases. They were in the core file — and in the Firefox version. But the file that actually got packaged into the Chrome zip was an older copy.

✅ Tests: green

The unit and integration tests ran against the core file. It had all the fixes. Everything looked perfect.

📦 Artifact: wrong

The zip contained a manual copy of the core — two releases old, missing the new fixes.

🚫 Users: affected

Everyone who downloaded the extension during that period got a product without the fixes the release notes promised.

How does something like this happen?

Classic cause: the same logic maintained in several places. When the project was small, the core was copied into each project by hand. The copy in the Chrome folder simply got forgotten for two iterations — while the source and the Firefox copy were updated.

Manual copies of code are time bombs. They work fine until someone forgets one. And there is no alarm when it happens.
Tests only test what they load. Our tests imported the core file directly. They knew nothing about what sat in the zip.
Release pressure amplifies it. Zips get built last, quickly, after everything is approved. That very step never gets the attention it deserves.

The fix: one source of truth + one parity test

No manual copies. The kernel logic now exists in exactly one file. The build script generates every copy — the extensions' scripts get the core spliced in at build time, using marked blocks.
Parity test before release. One test reads the finished zip, unpacks it, and compares the core section byte-for-byte against the source. If they differ, the build fails. No release without a green parity test.
Version bump before zips are built. Rule: any content change ⇒ version number up, always, before the artifact is built. A new zip can then never be mistaken for an old one.
Post-release verification. After deploying, the zip is downloaded live from the site and checked — the same parity check, but against what users actually download.

Three lessons you can use tomorrow

Test the artifact, not just the source. What users receive is the zip, the package, the installed thing. Your test suite should open exactly that at least once.
Composite projects should generate, not copy. If the same function exists in two files, they will diverge at some point. The question is not whether, but when — and whether a test notices.
The parity test is cheap and catches a whole class of bugs. It needs no framework: read two files, compare them, fail if they differ. Ten lines of code that saved us from shipping the wrong content a third time.

The mistake cost us two releases and some self-confidence. The fix took one afternoon. It is the best investment the project has made.

Try Clean Copy free → More guides

Frequently asked questions

What does release integrity mean?

That what you ship is identical to what you tested. Not "built from the same codebase" or "mostly the same" — but bit-for-bit the same files, same versions, no older artifacts mixed in. Integrity typically fails not in the code but in packaging: which files end up in the artifact.

Why didn't unit tests catch it?

Because they tested the source file. The core had all the fixes, and tests against the core passed. The failure lived elsewhere: in a copy of the code bundled into the zip, which no test ever read. A test can only guarantee what it actually loads.

What is a parity test?

A test that compares two things that should be equal — for example source core.js and the copy of core.js inside the release package. If they differ, the build fails regardless of whether functional tests are otherwise green. It catches exactly the class of bugs where something is maintained in two places.

How do I avoid this in my own project?

Three rules: (1) Generate copies from the source instead of maintaining them by hand — one source of truth. (2) Have the build script splice shared code into the artifact automatically. (3) Add one parity test verifying the artifact matches the source, and run it before every release.

Try Clean Copy free →