▲Production notes, written after the hard part
maria khan.
pages from a runbook
Most posts here began with something behaving differently in production than it did on paper. I write down the investigation while the wrong turns and useful details are still fresh.
- 9 posts
- 31 topics
- 5 mo writing
- 2026 since
A retry without a budget is a fan-out generator.
A lock without an owner token is a timeout with a race attached.
If a Redis key has no owner and no expiry decision, it is already an operational problem. It just has not paged anyone yet.
Read the library source when the runtime behavior and the documentation disagree.
A useful runbook preserves the wrong turns. The final fix alone is rarely enough to help during the next incident.
Shipped
I built leakferret after one too many secret findings arrived at the most expensive point, after the value had entered git history. Its Rust engine finds likely credentials, checks documented fixtures, verifies supported providers, and can rewrite a literal to an environment lookup. The same engine backs the CLI, editor extension, MCP server, and release packages.
10,400+ downloads across
- crates.io
- npm
- RubyGems
- Go
- VS Code
- Open VSX
- GitHub Actions
- MCP Registry
Reading shelf
- Designing Data-Intensive ApplicationsI went back for the chapters on replication and transactions. They made more sense after operating the failure modes.
- Site Reliability EngineeringThe incident response and capacity chapters still earn their place on the shelf.
- Working Effectively with Legacy CodeOlder than much of the code it would help with, and still painfully accurate.
- Database InternalsEnough detail to explain the machinery without pretending I am writing a storage engine.
- Understanding Distributed SystemsA compact map of the problems that become real only after something ships.
- Working with Ruby ThreadsThe GVL chapter remains the best starting point, even though Ruby gained several new concurrency tools.
- The Code BookI read this as a teenager. It was my first serious invitation into security.
Topics
Writing timeline
April 2026 hover a dot to see what published when August 2026
Recent writing
-
Redis keys need owners
A shared Redis instance had accumulated locks, claims, counters, and flags with no common ownership or expiry rules. A small registry turned those anonymous keys into named coordination primitives with explicit failure behavior.
-
Delayed delivery without unbounded queues
A fixed binary ladder turns arbitrary delivery delays into bounded RabbitMQ topology with message TTLs, dead-letter routing, and powers of two, without plugins.
-
Catching the secret before the commit, not after the audit
Rotating a credential after it entered git history kept costing hours. I built leakferret to catch the value earlier, separate fixtures from real findings, verify supported providers, and help remove the secret.