Circuit Breaker Pattern

What is the circuit breaker pattern?

The circuit breaker pattern is a resilience technique that stops a service from repeatedly calling a downstream dependency that is failing, giving the struggling dependency room to recover and preventing the failure from cascading. Named after electrical circuit breakers, it “trips” when it detects that calls to a service are failing consistently: once tripped (open), it immediately fails or falls back on subsequent calls without even attempting the downstream request, rather than making every caller wait for a timeout on a service that’s clearly down. It’s a foundational pattern for building robust distributed systems, and it works hand-in-hand with retries and timeouts.

Why it exists — preventing cascading failure

The problem the circuit breaker solves is cascading failure through retry amplification. When a downstream service degrades, naive callers keep sending requests (and often retrying failures), which piles load onto the already-struggling service, makes it worse, and ties up the callers’ own threads/connections waiting for timeouts — the failure spreads upstream as callers themselves become unresponsive, and one sick service takes down a whole chain. The circuit breaker interrupts this: after a threshold of failures, it opens and starts failing fast, so callers stop hammering the dead dependency and stop exhausting their own resources on doomed calls. The breaker moves through three states: Closed (normal — requests flow, failures are counted), Open (tripped — requests fail immediately without calling downstream, for a cooldown period), and Half-Open (after cooldown, a limited number of trial requests are allowed through to test whether the dependency has recovered — if they succeed, the breaker closes and normal flow resumes; if they fail, it re-opens). This gives the downstream time to heal and lets the system automatically recover without manual intervention.

Using it well — fallbacks and the broader resilience picture

A circuit breaker is most valuable when paired with a fallback: when the breaker is open, instead of just erroring, the caller can return cached data, a default response, a degraded experience, or a queued action — turning a hard failure into graceful degradation. It’s part of a family of resilience patterns that must work together: timeouts (a call must time out promptly, or a hung dependency ties up resources regardless of the breaker), retries with backoff and jitter (for genuinely transient blips — but the circuit breaker is what stops retries from becoming an attack on a truly-down service), and bulkheads (isolating resource pools so one failing dependency can’t starve everything). Circuit breakers are commonly provided by resilience libraries (Resilience4j, Polly) and by service meshes (Istio, Linkerd), which can apply breaking at the infrastructure layer without app code. The realities to get right: tuning the thresholds (failure count/rate to open, cooldown duration, half-open trial count) is workload-specific and matters — too sensitive trips on normal transient noise, too lax fails to protect; and the breaker needs good observability (you must know when breakers open, which is a key signal that a dependency is unhealthy). The recurring mistakes: implementing retries without a circuit breaker (so retries pound a dead service), circuit breakers without timeouts (hung calls tie up resources anyway), no fallback (a tripped breaker just errors instead of degrading gracefully), and untuned/unmonitored thresholds.

What people get wrong

• Retries without a circuit breaker — retrying against a truly-down service amplifies the outage; the breaker is what stops the pounding and lets it recover.
• No timeout or no fallback: without prompt timeouts, hung calls exhaust resources regardless of the breaker; without a fallback, an open breaker just errors instead of degrading gracefully.
• Untuned, unmonitored thresholds — trip settings are workload-specific (too sensitive trips on noise, too lax offers no protection), and open breakers are a critical health signal you must observe.

Primary source: Martin Fowler: CircuitBreaker