How to publish infrastructure work you can't fully benchmark

Two things are true at once for most independent infrastructure work. The ideas are testable — schedulers, cache layouts, policy engines are largely deterministic logic. And the hardware to test them at scale costs more per week than the project will ever return.

The two common resolutions are both bad. One is to publish a number anyway, produced on a synthetic harness or a single small model, phrased so a reader assumes it generalises. The other is to publish nothing measurable and lean on architecture diagrams. The first is a claim you cannot defend the moment someone asks how you measured it; the second reads as an idea rather than an engineering artifact.

There is a third option that costs almost nothing and almost nobody takes: state the verification status of every component, and let unverified mean unverified, in public, in the README.

Every non-trivial component in a systems project sits in one of three states, and the states are not interchangeable.

• Verified in CI. Runs on every commit, on commodity runners, with no special hardware. State machines, allocators, protocol encoders, config validation — anything with a CPU reference implementation.
• Verified on real hardware. Runs on a GPU runner or against a real dependency, on a schedule or on tags. Kernel correctness, engine integration, anything where the CPU path is a *model* of the real thing rather than the real thing.
• Wired but unvalidated. The interface exists, the types compile, the path is reachable — and no test has ever exercised it against reality. Multi-node transports, disaggregation fabrics, anything gated on hardware you do not have.

Putting that in the README as a table with a column for CI and a column for hardware costs about twenty lines and changes how the entire repository reads. A reader who sees explicit dashes in the hardware column starts trusting the checkmarks in the CI column. The dashes are what make the checkmarks worth anything.

More than most people assume. The claims worth making in an infrastructure project are frequently structural, and structure is arithmetic.

• Accounting against published constants. If a paper states a scheme costs 160 bytes per token per layer, a unit test can assert your implementation computes exactly 160. That test fails on a laptop the moment the layout is wrong. It is a real proof of a real property.
• Parity against a reference implementation. Write the algorithm twice — once in NumPy as an oracle, once as the fast path — and assert agreement to a stated tolerance across a spread of input sizes and seeds. The hardware run then only has to prove the port, not the algorithm.
• Property tests over invariants. Monotonicity, idempotence, ordering, conservation of allocated bytes. Hundreds of generated cases, no hardware.
• Chaos against mocks. Partitions, restarts, torn writes and checksum mismatches against a mock backend still exercise your recovery logic, because the recovery logic is the part that was ever in doubt.

What none of this buys is a performance claim. Throughput, latency, cache hit rate under contention, kernel occupancy — those need the hardware, full stop. The discipline is keeping the categories in separate sentences: a structural claim with a test reference beside it, a performance claim with a measurement or nothing at all.

Every project accumulates known gaps. The difference between a gap that reads as engineering judgement and one that reads as an unfinished thought is entirely presentation.

Give each one a stable ID, a source location, a severity and a target. *TD-035: the disk backend mirrors a host buffer; production should swap to zero-copy memory mapping behind a feature flag; Med; next sprint* is a decision. The same information as an inline // TODO: optimize is a smell. The ID is what lets it survive across sprints — you can close it, carry it, or explicitly defer it, and a reader can audit that you did exactly one of those.

The same logic covers decisions that never became debt. Recording why a lock-free queue lost to a short-held mutex, or why cache tier promotion is prohibited outright, costs a page and pre-empts an entire class of *why didn't you just—* review. Architecture decision records get pitched as a team artifact. They are at least as valuable solo, where you are the only person who will ever remember the reasoning, and you will not.

This one cost me time, so it is worth stating flatly. On one project, three passes of careful static review over a sprint's output found real issues and fixed them. Then I installed the toolchain and ran the full suite for the first time, and it found four more:

• A plugin whose classification state lived in a class attribute and was never populated — so the code read correctly and did nothing at runtime.
• Two individually correct tests sharing one config, where the mechanisms interacted and one test silently verified the wrong path.
• An assertion encoding my mental model of an eviction policy rather than its actual, correct behaviour.
• A build path advertised as dependency-free that in fact required a C toolchain.

None of those are visible on the page; all of them are visible in the first thirty seconds of a real run. Review verifies intent, execution verifies behaviour, and *the design is sound* is a statement about design only. The full write-up of that sprint is in the Meridian note.

There is a specific failure in how independent projects get presented. A repository carries CI badges, a documentation site, ADRs, a release with provenance — every surface marker of maintained software — and no statement of scope. A reader reasonably infers maintenance, files an issue, and gets silence. The signals that were supposed to demonstrate rigour become evidence of abandonment.

The fix is one paragraph in the README. This is a proof of concept, built in public, to demonstrate a specific thesis. These parts are verified, these are not. There is no maintenance commitment and no support expectation.

That paragraph costs nothing and makes everything else in the repository more credible, because the reader now knows you are the kind of engineer who marks the edges. A proof of concept that demonstrates capability is a completely legitimate artifact. A proof of concept dressed as a product is a claim you will eventually have to walk back, usually in front of someone whose opinion matters.

• A per-component verification table in the README, with a column for CI and a column for real hardware.
• Structural claims cited to the test that pins them.
• Performance claims omitted entirely unless you measured them, on stated hardware, with the harness published.
• A stated benchmark methodology — including what you deliberately do not measure.
• Tech debt with stable IDs, severities and targets, carried across sprints.
• ADRs for every load-bearing decision, including the alternatives you rejected and why.
• An explicit scope statement: proof of concept or maintained, and what support means.
• Licence, security policy and release provenance — the cheap parts of looking like you meant it.

None of this needs a GPU budget. All of it is more persuasive than a number the reader cannot reproduce.