Posts
Cell-based architecture as the middle ground between one dense cluster and one cluster per customer, and why moving data is the core capability.
Compartmentalization explains FoundationDB's architecture: find the responsibilities that got coupled together, split them, and scale only the parts that can scale.
Avro payloads cannot be decoded without the writer schema, and that two-schema model is what makes rolling upgrades and schema evolution work.
You fake the handful of methods your code actually calls, prove the fake stays honest, and then make it fail the way production does.
A retrospective on BugBash 2026, and why software engineering looks brighter than ever.
Trait-based fakes test partial failures that mocks and Testcontainers fundamentally cannot simulate. They are also the on-ramp to simulation-driven development.
How a 600-line Rust fuzzer discovers a 3-byte crash sequence in 9,000 iterations instead of 16 million
What I learned about manually constructing physical query plans for secondary index queries, and the library that came out of it.
Can LLM-generated code be trusted for distributed systems? Only if you have the right feedback loop.
How optimistic concurrency control works in FoundationDB, and how to design data structures that stop fighting it
LLMs amplify expertise, they don't replace it. Here's what works: planning, context, feedback loops, and building systems that let AI discover bugs.
Reflections on returning to engineering, discovering simulation as a superpower, learning to work with LLMs, and the compounding value of years of distributed systems work.
LLMs revived specifications. Model checkers like Fizzbee work for algorithms. But for APIs and business logic? The tooling doesn't exist yet.
Patterns and principles for writing Rust simulation workloads that catch bugs before production does.
Understanding how Kubernetes apiserver caches etcd, the 3-second timeout, and K8s 1.34 consistent read feature
How FoundationDB achieves legendary reliability through deterministic simulation, interface swapping, and one trillion CPU-hours of testing
How Rust's Deref trait solved a complex API design problem in FoundationDB-rs without traits or enums
A comprehensive guide on where and how to instrument metrics in your applications, covering everything from API endpoints to background jobs
Most testing prevents old bugs from returning. But what if we built systems where LLMs could actively discover new bugs instead?
The difference between shipped and operated software is the difference between something you can run and forget, and something that demands ongoing, hands-on care. Choosing the former protects your team’s focus and sanity.
Two podcast episodes—one from Oxide and one from Antithesis—on debugging at the limits and building correctness into systems from day one.
A review of Nix/NixOS after using it on all my machines for three years. I'll cover what works, what doesn't, and why it's the first OS that's stuck with me.
Why Apache DataFusion, a composable Rust-native SQL engine rather than a rigid framework, finally let me plug real queries into our FoundationDB-based system after years of watching from the sidelines.
A technical deep-dive into FoundationDB Record Layer continuations, explaining how they enable long-running operations by segmenting work across multiple FDB transactions, effectively bypassing the inherent 5-second and 10MB limits.
Discover lesser-known Tokio features like current-thread runtimes for !Send futures, seeded runtimes for deterministic tests, and paused time for precise temporal control in your Rust applications.
How deterministic simulation testing can help us build more reliable distributed systems and bridge the gap between development and production environments.
A curated collection of resources about deterministic simulation testing for distributed systems.
A key design trick for ordered key-value stores like HBase, FoundationDB, and RocksDB: reverse a number by subtracting it from its max value so a forward scan finds the highest one first.
A deep dive into the internals of FoundationDB's Data Distributor and how it manages shard placement and team priorities.
Strategies and techniques for enhancing safety in the FoundationDB Rust crate
I'm moving back in core engineering
Learn how to tune FoundationDB for Redwood storage Engine
Learn how to avoid FDB's biggest caveats by using a new feature called automatic idempotency in FoundationDB
Are you not using academic papers in your favorite RSS app? You should!
Learning distributed systems is tough. You need to go through a lot of academic papers, concepts, code review, before being able to have a global pictures. Thankfully, there is a lot of resources out there that can help you. Here's the best resources I used to learn distributed systems
An overview of all the helpers to generate row keys with FoundationDB
List of ressources gleaned about ETCD
A retrospective of the last 10 years
Opensourcing Record-Store
A code-level dive into how etcd achieves linearizable reads with the ReadIndex protocol on top of Raft, including how it enables safe follower reads.
List of ressources gleaned about Raft
We are excited to announce that StreamNative and OVHcloud are open-sourcing “Kafka on Pulsar” (KoP). KoP brings the native Apache Kafka protocol support to Apache Pulsar by introducing a Kafka protocol handler on Pulsar brokers
In today’s blogpost, we’re going to take a look at our upstream contribution to Apache HBase’s stochastic load balancer, based on our experience of running HBase clusters to support OVHcloud’s monitoring.
List of ressources gleaned about FoundationDB
A walkthrough of Kafka's wire protocol, from the API_VERSIONS handshake through producing and the consumer group coordination protocol, with a sequence diagram for each exchange.
A code-level dive into HBase's MemStore, from the ConcurrentSkipListMap backing HBase 1.x to the in-memory compactions Accordion introduced in HBase 2.0.
Piecing together, from scattered public talks and papers, how Google evolved GFS into Colossus, and why a Kubernetes-native distributed file system is still missing.
A hands-on walkthrough of HBase's TTL feature, how cell timestamps and compaction interact to expire data, and how to control it by setting your own version timestamp.
How OVH built Beacon, an Apache Flink pipeline handling alert deduplication, escalation, and notification on top of its Metrics Data Platform, and what running Flink in production taught us.
A walkthrough of the ACID properties of database transactions, from the 1983 Haerder and Reuter paper that coined them to isolation levels and PostgreSQL's MVCC implementation.
An overview of HBase's data model, row keys, column families and qualifiers, and why key design determines whether you get good performance or a hotspotted region.
A personal project importing NASA's Kepler and K2 telescope data into Warp10 and building a TensorFlow model to detect exoplanet transits from the dips in starlight.
An introduction to WarpScript, the stack-oriented language behind the Warp10 time series platform we ran at OVH's Metrics Data Platform to store and analyze sensor data.
An introduction to event-driven architecture: queue messaging systems applying the Unix pipe philosophy to microservices, and the simplicity, modularity, and scalability it buys you.
A look at what containers actually are, from Docker's write-once-run-everywhere hype to why a statically compiled Unix process, not a small VM, is the right mental model.