Balady Billing System
2024
Canonical microservices ensemble for government-integrated payment processing, points conversion, and wallet management — saga-style compensation, dual-verification on every callback, and circuit breakers on every external dependency.
The problem
Citizens use the Balady super-app for hundreds of transactional services — licences, certificates, infringements, fee payments. Every transaction has to hit the government payment gateway, reflect on the citizen’s wallet, settle reward points, and reconcile back to the originating service. Failures anywhere in that chain are visible to the citizen and audited by the bank.
The architecture
A focused ensemble of services grouped into payment-processing, wallet, and integration surfaces. Apigee fronts all ingress with rate-limiting and OAuth scoping. Saga-style compensation rolls back partial transactions when any leg fails. Dual-verification on every gateway callback ensures no settlement event is applied twice. Circuit breakers isolate failures of any single external dependency, and local fallbacks — cached authorisation decisions and policy snapshots — let the payment surface keep serving even when an upstream dependency is degraded.
Because a callback can arrive late or never, a scheduled reconciliation sweep detects completed payments whose callback never landed, with a back-office surface for manual recovery — so a missed webhook never strands a settlement. The saga approach is a deliberate trade of cross-service ACID guarantees for availability and isolation: each leg commits independently, compensating actions reverse partial work on failure, and a full audit trail is written on rollback.
Pattern view — saga compensation rolls back partial transactions; circuit breakers isolate each external dependency.
Data tier
Oracle 19c for transactional state (audit trail mandated by regulators). Redis Cluster for hot read paths (eligibility, wallet balance) — authorisation sits on the hot path of every wallet operation, so decisions are cached and refreshed out-of-band from the policy source, keeping reads fast while bounding how stale a decision can be. Apache Kafka carries asynchronous settlement events to the analytics tier and to sibling services that consume wallet and settlement events.
Outcome
The canonical payments substrate for the Balady ecosystem. Consuming services integrate via the same published event stream, so changes propagate uniformly.