Patterns — a map, not a pile

These docs were written one at a time as each pattern was built. Read on their own they look like 13 unrelated topics; this page is the thread that connects them.

Every doc follows the same shape: the problem → the pattern → where it lives in our code → trade-offs → interview Q&A → "try to break it". The point is to be able to explain each one and to know what it costs, not just to recognise the name.

The one-sentence story

A client sends one request through a gateway that proves who they are; that kicks off a saga that moves money across two services by passing events over RabbitMQ; each step writes its state and its outgoing event atomically and tolerates duplicate delivery; the account service stores that state as an event stream and serves reads from a separate CQRS read model; and the whole flow is observable end-to-end by one correlation id.

Each bold word is a doc below.

By topic

1 · Moving work between services reliably

The hard part of async isn't sending messages — it's not losing or double-applying them when things crash.

• transactional-outbox — write state and the outgoing event in one transaction, so you can never do one without the other (the "dual-write" problem). Account-service later outgrows this — see §6.
• idempotent-consumer — delivery is at-least-once, so duplicates are normal; dedupe on messageId in the same transaction as the work.
• retry-dlq — transient failures retry with backoff; poison messages land in a dead-letter "parking lot" instead of blocking the queue forever.

2 · A distributed transaction without a distributed transaction

• saga-choreography — there is no 2-phase commit across databases; the transfer is a chain of local steps, and failures compensate (undo via a new forward action) rather than roll back. This is the spine of the whole system.

3 · The domain, and the contract between services

• ddd-bounded-contexts — why two services at all, what the Account aggregate owns, and database-per-service.
• event-contracts-versioning — the event envelope is a public API; how it's shaped and how it's allowed to evolve.
• correlation-and-logging — one id per saga threaded through every log line, via AsyncLocalStorage / log context.

4 · The edge: one front door, zero trust

• api-gateway-yarp — a single public entry point that handles routing, CORS, rate-limiting, and correlation-id minting once, at the edge.
• oidc-keycloak-zero-trust — OIDC + PKCE login, and why the JWT is validated at the gateway and again inside every service.
• bulkhead-load-shedding — protecting the edge from slow downstreams via concurrency limits and fast-fails.

4.5 · Resilience Testing

• chaos-drills — formalized fault injection scenarios to prove that resilience patterns (circuit breakers, outbox, DLQs) actually work under duress.

5 · Seeing what happened

• distributed-tracing-otel — OpenTelemetry traces, and the honest finding that the async hop breaks the live trace (and what to use instead: the correlation id).
• metrics-slos-grafana — turning "is it healthy?" into numbers: SLIs, SLOs, the saga success-rate and P99-vs-5s dashboard, the parking-lot alert.

6 · Data architecture: events as the source of truth

This is where the account service stops storing state and starts deriving it.

• event-sourcing — the balance is a fold of events, not a stored number; snapshots, optimistic append, and the elegant consequence that the event store replaces the outbox from §1.
• cqrs-read-models — split the write model from the read model; serve reads from Redis views that are rebuildable from the event log; embrace eventual consistency. Note that CQRS ≠ event sourcing — the two pair up here but are independent.

Suggested reading order

If you're reviewing for an interview, read top-to-bottom: §1 → §2 are the core (outbox, idempotency, saga), then §3 for vocabulary, then §4–§5 for the production concerns, and finally §6, which revisits §1 from a higher vantage point ("now that events are the state, do we still need an outbox?").

How the big ideas connect

• The outbox (§1) exists because state and events were two writes. Event sourcing (§6) makes the append the only write, so the store becomes the log and the outbox dissolves — same guarantee, fewer moving parts.
• Idempotent consumers (§1) are what make at-least-once delivery safe, which is exactly why the event-store-as-outbox relay (§6) can re-publish freely.
• The saga (§2) is choreographed by the events that §1 delivers and §3 versions.
• Correlation id (§3) is the pivot that tracing (§5) falls back on when the async hop breaks the span — the same id that ties §1's logs together.
• CQRS (§6) is why a slow analytical read never touches the write path that the saga (§2) depends on.