Nick Cerutti

AI Architect & Consultant · Production LLMs & Agents

I design and harden the systems that break after the demo. Agents, inference, and multi-tenant platforms that survive contact with real users and real budgets.

Work Experience Projects Notes Resume

Open to new engagements

Book a 15-min call

Experience

Professional experience

Systems, infrastructure, and advisory work across trading, production AI, and distributed engineering organizations.

Brought in across 6+ fast-growing platforms, from Series B to late-stage teams, to build the multi-tenant substrate for agent systems serving 250-450 concurrent instances across AWS Bedrock, AgentCore, and Azure environments.
Led incident response for a production multi-agent cost explosion projected at $40K+ per day, implementing circuit breakers, per-session spend caps, and retrospective governance controls within 72 hours with zero recurrence.
Defined Service Level Objective (SLO) frameworks across 3 client platforms using task completion rate, mean tool calls, hallucination escape rate, and end-to-end latency; improved SLO compliance from 71% to 96% over two quarters.
Audited and built retrieval and governance systems supporting 1.8M+ tool invocations, with deterministic policy validation and zero known bypasses across deployments using Pinecone and OpenSearch.
Built Go, Python, and TypeScript middleware, Azure DevOps templates, and deployment automation for 4 distributed engineering teams, reducing release-cycle overhead by roughly 40%.
Refactored client integrations with Anthropic SDK, n8n, and Copilot Studio to enforce token optimization, caching, and low-infrastructure patterns, delivering an aggregate $200K annual reduction in cloud and LLM spend.
Standardized data lakes and model registries across 2 client platforms, integrating Databricks and Domino Data Lab with classical ML workloads in scikit-learn, PyTorch, and TensorFlow.

Designed and ran in production a real-time trading system for prediction markets, using event-driven Python and Rust pipelines to pull live data from centralized exchanges and on-chain sources while holding sub-10ms latency at peak load.
Designed the risk-control layer for live trading with position limits, daily loss limits, and slippage checks, cutting execution slippage by 14% with zero limit breaches while trading real capital.
Built statistical models of market order flow to detect volatility clustering and used them to size trades and set execution pace under fast-moving conditions.
Built Kelly-criterion-based position sizing that scales exposure to model confidence, reducing tail losses in high-uncertainty periods without giving up long-run expected value.
Maintain a research pipeline covering historical backfill, detector-faithful backtesting, walk-forward validation, and automated signal outcome labeling.

Contributed to a seven-engineer MLOps pod, spearheading the enterprise deployment and evaluation infrastructure while ensuring pipeline reliability across containerized LLM and inference workloads in production.
Deployed and operated a multi-GPU Triton Inference Server cluster with dynamic batching and model quantization for NLP and Computer Vision pipelines, defining inference SLOs and Prometheus/Grafana alerting runbooks for OOM failures.
Led the decomposition of a monolithic FastAPI inference service into event-driven microservices with Docker, PostgreSQL, and Kafka, reducing cold-start deployment time from 18 minutes to under 4.
Owned schema and indexing strategy for behavioral analytics datasets used by 3 cross-functional product teams, driving p99 query latency from over 900ms to 120ms through index and materialized-view redesign.
Engineered GitOps-driven CI/CD with GitHub Actions and ArgoCD, automated health checks, and canary rollout policies, sustaining 99.9% uptime across production containerized environments.

Assisted the core infrastructure team in containerizing legacy monolithic applications with Docker for smoother development and staging transitions.
Wrote Python and Bash automation scripts to clean, parse, and ingest semi-structured log data into staging databases, accelerating debugging workflows.
Configured Prometheus and Grafana monitoring for server health, memory allocation, and API latency, while optimizing relational schemas and SQL queries for internal analytics dashboards.

Client work

Production work, with the numbers left in.

Eight client work examples — seven anonymized, one public (Iguazio). Every metric below also appears in the experience section above.

Production emergency

$40K+/day burn contained within 72 hours. Zero recurrence.

View case

Multi-tenant agent substrate

250-450 concurrent agent instances in production. Release overhead down roughly 40%.

View case

LLM cost reduction program

$200K/yr aggregate reduction in cloud and LLM spend.

View case

See all 8 work examples

Projects

Selected open-source work

Tooling for agent memory, governance, orchestration, and the operational realities around production deployment — built in public at github.com/angelnicolasc.

Graymatter

Drop-in persistent memory for AI agents. Cut token consumption by 97% with a single zero-dependency binary. No Docker, no external databases, no cloud storage. Pure pragmatism. +450 GitHub stars and 35 forks.

Golang / Agent Memory / Hybrid Retrieval / MCP Server / Persistent State

Highlights

Built a storage layer using bbolt for facts and metadata plus chromem-go for vector search, with durable reconciliation and background reindexing.
Implemented hybrid recall using vector similarity, keyword scoring, recency weighting, and reciprocal rank fusion tuned for prompt injection into live agent runs.
Built ingestion through direct remember flows, extraction-assisted memory writes, agent self-curation, and asynchronous consolidation with decay and pruning.
Shipped a full MCP server with auto-onboarding, memory search, checkpointing, and real-time Bubble Tea observability dashboards.

Epica

Embeddable Rust library for formal causal belief revision in LLM agents. Grounded in 5 arXiv papers, it delivers dual-process uncertainty quantification and cryptographically auditable trails.

Rust / PyO3 / Causal Graphs / Agent Governance / MCP

Highlights

Built an embeddable 12-crate Rust runtime implementing formal AGM contraction and semantic-equivalence postulates to enforce contradiction-aware belief updates across tool cycles.
Implemented dual-process uncertainty via System 1 Noisy-OR causal propagation and System 2 asynchronous LLM reflection, derived directly from foundational variational free-energy research.
Developed a tamper-evident audit ledger using BLAKE3 Merkle hash chains, enabling decentralized third-party verification of agent memory and governance history via an offline CLI tool.
Engineered a native MCP server with 16 routes supporting OAuth 2.1 authentication and Prometheus telemetry, exposing Python SDK bindings compiled via PyO3.

Meridian

Inference-time compute scheduler for reasoning models, optimizing token-generation phases via dual-queue dispatch and entropy-driven budget control.

Rust / Python / CUDA / Inference Infrastructure / vLLM

Highlights

Designed and engineered a phase-aware vLLM scheduling layer for reasoning models including DeepSeek-R1, Qwen3, and o3 to decouple think-decode and output-decode workloads at runtime without codebase forks.
Implemented a dual-queue dispatch system alongside a three-tier phase-aware KV cache eviction mechanism, neutralizing inter-token latency bottlenecks caused by heavy reasoning batch dynamics.
Integrated entropy-driven budget forcing leveraging EAT and RPDI signals via custom CUDA kernels running on secondary GPU streams to dynamically inject termination boundaries based on convergence.
Developed a drop-in vLLM plugin architecture using reversible attribute delegation, adding disaggregated KV transfer hooks to interface with high-performance prefill-decode fabrics.

Stratum

Local dual-tier LLM serving stack featuring sub-millisecond adaptive complexity routing and real-time VRAM orchestration.

Python / vLLM / llama.cpp / Infrastructure Optimization / ASGI

Highlights

Designed and deployed a dual-tier LLM serving stack capable of supporting 200 concurrent users via optimized container orchestration.
Built an ASGI-compatible adaptive proxy middleware executing sub-millisecond multi-dimensional routing based on prompt complexity, SLA latency targets, and hardware availability.
Developed an asynchronous hardware monitoring system utilizing raw NVML bindings to track GPU state and dynamically enforce real-time circuit-breaking policies under load.
Implemented benchmarking harnesses for prefix caching evaluation and multi-user token-throughput stress testing across heterogeneous engine setups.

Drako

Open-core security and governance platform for production AI agents, spanning static analysis, policy authoring, and deterministic runtime enforcement.

Python / Policy-as-Code / Runtime Governance / AI Agent Security

Highlights

Designed and implemented a 97-rule, zero-LLM policy engine across security, compliance, and FinOps, achieving 100% deterministic evaluation with sub-2ms overhead (<0.3% of an LLM call).
Built static analysis features featuring reachability analysis to eliminate false positives, Agent BOM generation, baseline tracking, and native GitHub Actions/SARIF CI integration.
Built a 13-stage production runtime pipeline covering Presidio-based Data Loss Prevention (DLP), cryptographic audit trails, circuit breakers, spend controls, and EU AI Act (Art. 14) human-in-the-loop review.
Added autopilot config generation, behavioral drift detection, and 100% offline desktop MCP scanning for Cursor and Claude with framework guardrails for LangGraph, CrewAI, and AutoGen.

Forge

Universal harness that wraps existing agent systems to add observability, memory, budget controls, and safe self-evolution without forcing framework migration.

Python / Multi-Agent Orchestration / Self-Optimizing Runtime / Observability

Highlights

Designed adapters for LangGraph, CrewAI, AutoGen, and generic async callables on top of a shared runtime and event bus.
Implemented OpenTelemetry tracing, per-agent and per-model cost tracking, real-time budget enforcement, and REST plus SSE monitoring APIs.
Developed cross-run memory that combines vector retrieval, knowledge graph persistence, provenance tracking, and conflict-aware versioning.
Built a snapshot-safe self-evolution loop for prompts, parameters, models, and topology with rollback and A/B safeguards.

More experiments on GitHub

Work with me

Production-minded engagement.

Three common starting points. Nine consulting services in total.
Start from what is actually broken — each of these opens the engagement built for it. The full list is one click below.

Spend is growing faster than usage.

LLM Cost & Routing Sprint

3 weeks · $15-20K

Agents are in production and nobody can say whether they are safe.

Agent Production Audit

2 weeks · from $12K

The team is shipping AI and nobody owns the architecture.

Fractional AI Architect

3 months minimum · $9-15K/mo

Consulting servicesWhat each costsClient workEight free tools

Agents burning money right now? Email me directly — skip the calendar.

If you're building something where AI has to work in production, let's talk.

Book a 15-min call

No deck, no pitch. Not sure which one fits? Send me what you're building and you'll get a straight read back — scope, sequence, and where a $2,500 pilot makes sense before anything bigger.

On the employment track, I'm also open to full-time Head of AI and AI Architect roles. Download the resume or ask in the call.

Notes

Engineering notes

Short technical notes from real infrastructure work. The kind of details that usually stay inside private Slack channels or incident reviews. Last updated August 8, 2026.

Your security team says this fix is free. It costs between $721 and $61,000 a year.

Build notes from KVSentry: why per-tenant cache isolation is not the zero-cost mitigation it gets quoted as, how to price it from lost cache hit rate instead of a published percentage, and the one metric that moves the answer by two orders of magnitude.

FinOps / KV Cache / Multi-Tenant Security / vLLM

Aug 8, 20269 min read

The Cost You Can't See

Itemising what one architectural component of a 284B model actually costs to serve - and why the published description of it was wrong by 2x.

Inference / Cost Modeling / Open Source / DeepSeek-V4

Aug 7, 20266 min read

How to publish infrastructure work you can't fully benchmark

Most infrastructure side projects are written on a laptop and benchmarked on a slide. A per-component verification column is cheaper than a bigger claim, and worth more.

Engineering Practice / Open Source / Benchmarking / Architecture Decision Records

Aug 4, 20268 min read

See all 5 notes

Capabilities

Technical focus

Broad enough to span research, infra, and runtime operations.
Specific enough to ship in production.

Languages & Runtimes

Core implementation languages for systems work, infrastructure tooling, and performance-sensitive runtime paths.

Go, Python, TypeScript, Rust, SQL, C++, CUDA.

Agent Systems

For orchestrating multi-step agent behavior, memory, tool use, and compound execution across frameworks.

Multi-agent orchestration, A2A protocol, MCP server and client development, Context engineering, Persistent and episodic memory systems, Hybrid retrieval-augmented generation (RAG), Tool-use and function-calling pipelines.

Inference & Serving

From single-GPU deployments to multi-tenant inference clusters under real production load.

VLLM, Triton Inference Server, Model routing, AWQ and GPTQ quantization, KV-cache optimization, Prompt caching, Structured output contracts, Low-Rank Adaptation (LoRA) and Quantized Low-Rank Adaptation (QLoRA) pipelines.

Scale & Reliability

For systems that need to stay observable, cost-aware, and resilient as traffic, tenants, and model complexity grow.

Multi-tenant agent platforms, Kubernetes at scale, Service Level Objective (SLO) and error budget design, Incident response, Chaos engineering, Capacity planning, OpenTelemetry, Prometheus and Grafana, Agent trace instrumentation, Real-time memory analytics, Token spend tracking, Audit trails, Evaluation loops.

Edge & Constrained Environments

When Kubernetes is too much and reliability still is not optional.

Zero-dependency binaries, Embedded storage, Offline-first agents, Single-binary MCP servers, Bubble Tea observability, WASM-compatible tooling.

Security & Governance

Enforcement that runs at runtime, not just on paper, across agent policies, compliance, and tool-call controls.

Policy-as-code, Runtime tool-call validation, Deterministic evaluation loops, Reachability analysis, Agent BOM, Desktop agent scanning, Prompt injection mitigation.

Delivery & Automation

For shipping changes repeatedly, enforcing standards, and reducing manual operational overhead.

GitHub Actions, ArgoCD, MCP integrations, CLI tooling, Pre-commit hooks, SARIF, GitOps pipelines.

Education

Academic background

Computer science, business, and applied ML training with an emphasis on systems thinking.

S21 University

B.S. in Computer Science

GPA 3.92

Buenos Aires, Argentina

Coursework

Data Structures and Algorithms, Advanced Operating Systems, Software Engineering, Database Systems, Computer Architecture, Entrepreneurial Development.

Jan. 2018 - May 2021

IAE Business School

Master of Business Administration (in progress)

Buenos Aires, Argentina / Online

Coursework

Corporate Finance, Strategy & Business Model Innovation, Financial Accounting, Operations & Logistics, Leadership & Human Behavior.

Feb. 2025 - May 2027

DeepLearning.AI / Coursera

MLOps Specialization

Online

Coursework

Introduction to Machine Learning in Production, Machine Learning Data Lifecycle in Production, Machine Learning Modeling Pipelines in Production, Deploying Machine Learning Models in Production.

Mar. 2024

Nick Cerutti

Professional experience

AI Infrastructure & MLOps Engineer

Quantitative Systems Engineer

AI Infrastructure & MLOps Engineer

Junior Infrastructure & Data Intern

Production work, with the numbers left in.

Selected open-source work

Graymatter

Epica

Meridian

Stratum

Drako

Forge

Production-minded engagement.

Engineering notes

Your security team says this fix is free. It costs between $721 and $61,000 a year.

The Cost You Can't See

How to publish infrastructure work you can't fully benchmark

Technical focus

Languages & Runtimes

Agent Systems

Inference & Serving

Scale & Reliability

Edge & Constrained Environments

Security & Governance

Delivery & Automation

Academic background

S21 University

IAE Business School

DeepLearning.AI / Coursera