Centralized DB Decomposition
2024
Platform-level proposal decomposing a shared database into per-business-domain stores via Debezium CDC + Kafka, with phased migration and Grafana / Prometheus monitoring.
The problem
A single shared operational database had grown to be the bottleneck of the platform. Schema changes by one team rippled across every other team sharing the database; reporting queries contended with operational ones; the database was the implicit coupling layer between everything.
The architecture
A phased decomposition strategy: each business domain gets its own database, and Debezium Change Data Capture streams row-level changes into Apache Kafka, where a configurable transformation-and-routing layer reshapes and directs each change to the right per-domain store — letting the legacy shared database fade out.
A deliberate choice was to make that transformation-and-routing layer business-configurable: replication rules and schema mappings are defined through a low-code interface rather than hand-written per migration, so the platform is reusable across many domain cut-overs instead of becoming a one-off ETL job. Because change events flow asynchronously, a conflict-resolution step reconciles late or competing writes before they reach a per-domain store, keeping the new databases consistent during the overlap window when both old and new paths are live.
ADRs govern the design:
- Kafka over RabbitMQ for the CDC bus — Debezium ships first-class Kafka connectors, and Kafka’s retention semantics suit replay-able change streams
- A low-code, business-configurable pipeline layer — domain teams define replication and transformation rules through a UI rather than waiting on engineering
- Grafana + Prometheus for the unified observability layer
Pattern view — Change Data Capture streams a shared database into per-domain stores; the monolith DB fades out.
Migration approach
For each domain: stand up the new database → backfill from the source → flip the read side first → flip the write side once parity is verified → retire the legacy schema slice. The Strangler Fig pattern applied to data instead of services.
Outcome
A platform-level path out of the centralized database without big-bang risk. Per-domain teams gain schema autonomy. The design is intentionally generic — the same platform can drive any future per-domain decomposition rather than being tied to a single migration.