Notes
Write-ups from building inference infrastructure in public: scheduling reasoning models as two workloads, what MLA and DeepSeek-V4 do to KV cache block layout, and how to publish systems work honestly when you cannot afford to benchmark all of it.
Written by Nick Cerutti, AI infrastructure architect. Last updated . RSS.
Most infrastructure side projects are written on a laptop and benchmarked on a slide. A per-component verification column is cheaper than a bigger claim, and worth more.
Engineering Practice / Open Source / Benchmarking / Architecture Decision Records
Build notes from Tessera: what a KV block manager has to change when the cache is a 56× compressed latent instead of full K/V, what changes again when attention goes per-layer hybrid, and why content-addressed dedup needs byte verification.
KV Cache / Multi-head Latent Attention / DeepSeek-V4 / vLLM
Build notes from Meridian: why a reasoning request should be scheduled as two workloads with separate SLOs, how entropy replaces the token timer, and what broke when I ran the toolchain instead of reading the code.
Inference Infrastructure / vLLM / Reasoning Models / KV Cache