Notification Hub

The problem

Citizens receive notifications about licences, fee deadlines, and service status from many different Balady services. Without a hub, every service implements SMS / push / web-toast itself — inconsistent templates, no central rate-limit, no shared retry semantics, no audit.

The architecture

A staged pipeline:

1. Ingest — producer services publish a notification request (recipient, template id, channel preferences, locale).
2. Templating — a dedicated templating service renders the request through the appropriate template: localised content with dynamic token substitution. Templating is kept as a separate concern from dispatch, so adding or changing a message template never touches the delivery path.
3. Persist-before-dispatch — the rendered notification is written to the MongoDB notification store before dispatch attempts begin. This guarantees we never deliver-and-forget; every send has a durable record. Because the notification is persisted before any send is attempted, every failed delivery already has a record to retry from — persistence is what makes safe, bounded retry possible rather than a separate bookkeeping concern.
4. Per-channel dispatch — separate RabbitMQ queues per channel (SMS, mobile push, web). Channel-level back-pressure cannot poison other channels.

Pattern view — persist-before-dispatch pipeline with per-channel queue isolation.

Retry is driven asynchronously off the durable store rather than inline in the dispatch path: a scheduled sweep re-enqueues notifications whose last attempt failed, so a temporarily unavailable provider self-heals without blocking new traffic. Retry is bounded — a fixed maximum number of attempts per notification, with exponential backoff between them. Permanently failed deliveries are flagged in the store so they can be surfaced for follow-up.

Outcome

One notification model for the entire Balady ecosystem. Per-channel queue isolation means an SMS-vendor outage doesn’t slow push notifications, and vice versa.