Load Balancer

What is a load balancer?

A load balancer spreads client requests across a pool of backend targets so no single server bears all traffic. It improves availability (if one server fails, others continue) and scalability (add more targets behind the same entry point).

Layer 4 vs Layer 7

Type	Operates at	Typical use
Layer 4 (NLB)	TCP/UDP	Raw throughput, gaming, databases
Layer 7 (ALB)	HTTP/HTTPS	Path-based routing, host headers

Common algorithms

• Round robin — rotate through backends evenly
• Least connections — send to least busy server
• IP hash — sticky sessions based on client IP

Health checks

Load balancers probe backends (/health) and stop sending traffic to unhealthy instances. This is essential for zero-downtime deploys and auto-scaling groups.

Cloud examples

AWS Application Load Balancer, Network Load Balancer, Google Cloud Load Balancing, Azure Load Balancer, and HAProxy / NGINX self-managed options.

Relationship to API Gateway

Load balancers focus on traffic distribution. API gateways add API policies. In large architectures, traffic often flows: Client → CDN → Load Balancer → API Gateway → Services.

Cost reality

Managed load balancers bill hourly plus per-GB processed, and the hourly part accrues on idle ones too — orphaned ALBs from deleted environments are a classic line item in every cloud-bill cleanup. At high throughput, the per-GB charge can exceed the compute behind it; that’s when teams evaluate HAProxy/NGINX on their own instances.

What people get wrong

• Health checks that lie. A /health that returns 200 while the database connection pool is dead keeps routing traffic into failures; health checks should verify dependencies, shallowly.
• Forgetting connection draining. Deploys that kill instances without draining in-flight requests turn every release into a small outage.
• Sticky sessions as a crutch. Session affinity hides state problems; the fix is externalizing session state, not pinning users to servers.