Event-driven Architecture - Summary

Overview

#

• Event-driven architecture (EDA) is a distributed, asynchronous style built from decoupled event processors that receive and react to events.
• Suitable for both small and large systems; often embedded inside other styles (e.g., event-driven microservices).
• Contrasts with the request-based model (synchronous, deterministic request → orchestrator → processors).

Topologies

#

Broker Topology

#

• No central controller; message flow is broadcast/chain-like through a message broker (topics/exchanges).

• Core components: initiating event, event broker, event processor, processing event.

• Processors consume initiating events, perform tasks, then publish processing events that other processors may consume.

• Characteristics

  • Very decoupled — producers don’t know consumers.
  • High responsiveness, scalability, and extensibility (easy to add new consumers).
  • Uses publish/subscribe (topics) for fire-and-forget broadcast.

• Drawbacks

  • No global workflow control or transaction owner → hard to know when a business transaction is complete.
  • Error handling and recoverability are difficult; failures can leave the system in inconsistent states.

Mediator Topology

#

• Central event mediator controls workflow, state, acknowledgements and error handling.

• Components: initiating event, initiating event queue, event mediator, event channels (queues), event processors.

• Mediator sends commands (point-to-point) to processors; processors respond back to mediator with acknowledgements.

• Characteristics

  • Strong workflow control, easier error handling and restart/recoverability.
  • Processors don’t broadcast what they did; mediator orchestrates.

• Drawbacks

  • Harder to model dynamic flows declaratively; mediator scaling can become a bottleneck.
  • Less decoupling and somewhat lower performance versus broker topology.

• Hybrid approach: use a simple mediator to classify events and delegate to more complex mediators (BPEL/BPM/Step Functions) when needed.

Asynchronous Capabilities

#

• EDA relies on asynchronous communication for both fire-and-forget and request/reply (pseudo-sync) patterns.
• Asynchrony improves responsiveness (user sees immediate acceptance) but does not automatically improve end-to-end performance.
• Example: posting a comment — synchronous path = user waits for full processing; asynchronous path = user gets quick ack while processing continues.

Error Handling — Workflow Event Pattern

#

• Workflow event pattern (reactive) improves resiliency without harming responsiveness through a workflow delegate:

  • Consumer delegates errors immediately to a workflow processor and moves on to next message (preserves responsiveness).
  • Workflow processor attempts programmatic repair (or ML-based remediation); if repair fails, it routes the message to a human-facing dashboard for manual fix and resubmission.

• Example: trading basket with malformed field ("8756 SHARES") → TradePlacement delegates error → TradePlacementError service strips "SHARES" and resubmits.

Ordering and Recoverability

#

• Repair/resubmit may process messages out of sequence. For contexts where order matters (per account), implement holding queues (per account FIFO) until the errored message is fixed, then resume processing in order.

Preventing Data Loss

#

Three common loss points and mitigations:

1. Producer → Queue: use persistent queues + synchronous send (broker persists message) → guaranteed delivery.
2. Queue → Consumer crash: use client-acknowledge / visibility timeouts so messages are not removed until processing succeeds.
3. Consumer → Database write failure: use ACID transactions and Last Participant Support (LPS) semantics — only remove message from queue after DB commit succeeds (consumer deletes message afterward).

Broadcast Capabilities

#

• EDA supports broadcasting without producer knowledge of subscribers (highest decoupling).
• Useful for eventual consistency, CEP, and scenarios like market tickers where many consumers react to the same data.

Request-Reply (Pseudo-synchronous)

#

• Implemented over asynchronous channels using two queues: request queue + reply queue.

• Two common techniques:

  1. Correlation ID: producer records request MessageID; consumer sets CorrelationID = MessageID in reply; producer filters on CID.
  2. Temporary reply queue: create per-request reply queue and pass its address in reply-to; simpler but costly at high volume.

• Producers either block waiting for reply (with timeout) or use async callbacks/polling.

Choosing Between Request-Based and Event-Based

#

• Request-based: use when requests are well-structured/data-driven and you need determinism, control and immediate responses.
• Event-based: use when you need responsiveness, scale, flexibility, and dynamic processing.
• The choice is a trade-off: workflow/control & recoverability (mediator/request) vs extreme decoupling, throughput & scalability (broker/event).

Hybrid Architectures

#

• EDA commonly combined with other styles (microservices, space-based, microkernel, pipeline).
• Benefits: reduces bottlenecks, enables back pressure, improves responsiveness and programmatic scalability.
• Use messaging as data pumps and interservice communication.

Architectural “Quanta” and Characteristics

#

• A quantum groups processors that share synchronous ties (e.g., same DB instance or immediate request-reply dependency).

• EDA rates high (★★★★★) in performance, scalability, fault tolerance, and evolutionary change:

  • Performance via async + parallelism.
  • Scalability via competing consumers and programmatic scaling.
  • Fault tolerance via decoupling and eventual consistency.

• Lower ratings for simplicity and testability due to nondeterministic, dynamic event flows and complex event trees.

Practical Guidance / Patterns

#

• Use broker topology for highly scalable, loosely coupled broadcast scenarios and extensibility.
• Use mediator topology (or hybrid) when you need workflow control, error handling, and recoverability.
• Apply the workflow event pattern to keep responsiveness while handling errors programmatically.
• Employ message durability, ack/visibility timeout, and transactional DB commits (LPS semantics) to avoid data loss.
• Prefer Correlation ID for request-reply at scale; temporary reply queues only for low volume or simple cases.