Designing a resilient service architecture
[Fixture] A target architecture that let a growing product survive the loss of a zone without downtime.
Context
[Fixture] A fictional online booking product.
The problem
Every service ran in one place, so a single infrastructure failure took the whole product offline.
My role
Architect responsible for the target design and its rollout plan.
Approach
Mapped which services depended on which, and which could tolerate delay.
Spread services across two zones, with a queue between the parts that did not need to answer immediately.
Replicated the database between zones and rehearsed failover before relying on it.
Outcome
A zone failure now degrades the product instead of stopping it.
Stack
- Cloud load balancing
- Message queue
- PostgreSQL
- Terraform