Notes
Write-ups from building inference infrastructure in public: scheduling reasoning models as two workloads, what MLA and DeepSeek-V4 do to KV cache block layout, how to publish systems work honestly when you cannot afford to benchmark all of it — and the costs nobody prices.
By Nick Cerutti — AI Architect. Last updated . RSS.
Build notes from KVSentry: why per-tenant cache isolation is not the zero-cost mitigation it gets quoted as, how to price it from lost cache hit rate instead of a published percentage, and the one metric that moves the answer by two orders of magnitude.
FinOps / KV Cache / Multi-Tenant Security / vLLM
Itemising what one architectural component of a 284B model actually costs to serve - and why the published description of it was wrong by 2x.
Inference / Cost Modeling / Open Source / DeepSeek-V4
Most infrastructure side projects are written on a laptop and benchmarked on a slide. A per-component verification column is cheaper than a bigger claim, and worth more.
Engineering Practice / Open Source / Benchmarking / Architecture Decision Records
Build notes from Tessera: what a KV block manager has to change when the cache is a 56× compressed latent instead of full K/V, what changes again when attention goes per-layer hybrid, and why content-addressed dedup needs byte verification.
KV Cache / Multi-head Latent Attention / DeepSeek-V4 / vLLM
Build notes from Meridian: why a reasoning request should be scheduled as two workloads with separate SLOs, how entropy replaces the token timer, and what broke when I ran the toolchain instead of reading the code.
Inference Infrastructure / vLLM / Reasoning Models / KV Cache