Event-driven architecture or Designing event driven systems

Event-driven architecture is a software architecture and model for application design in which the behavior of the software is determined by events, such as clicks, sensor data or other types of messages. Rather than processing data in closed batches, event driven systems process data continuously in real time as it streams into the system.

How to design Event Driven Systems

Designing event driven systems requires a careful understanding of how the various components interact and how time is measured and handled across the various parts of the system.

An event driven system is generally divided into event producers and event consumers. The producers generate data whenever specific 'events' occur. This could be a click on a website or a credit card transaction at a restaurant. The event consumers register the incoming data and process it according to the logic defined in the system.

What is Event Stream Processing (ESP)?

Event Stream Processing, or ESP, is the practice of performing computations on streaming data in real time. Platforms like Pathway offer dedicated code solutions to perform this task, designed specifically for this purpose. Event Stream Processing is used in range of applications, such as IoT, supply chain, finance, and any application that produces clickstream data.

What is the difference between Event Driven Systems and Event Stream Processing?

Event Driven Systems are the actual, implemented architectures that performs Event Stream Processing. ESP is generally used to refer to the general practice of processing event-driven data in real time. Event Driven Systems is generally used to refer to the specific implementations and software architectures.

Can Event Driven Systems also work with historical (batch) data?

Most conventional Event Driven Systems are not able to natively process historical batch data. Often, code rewrites are required in order to transpose computational logic from a batch context into a streaming application (or vice versa).

Pathway is an example of a unified platform that can perform both types of operations (streaming and batch) using the same code syntax. Unified platforms like Pathway provide extra features not present in traditional ESP platforms, including support for synchronous interactions with request/reply mechanisms and the capacity to handle long-term reference and state data.

Gartner's 2023 Market Guide for Event Stream Processing

Gartner recently published its 2023 Market Guide for Event Stream Processing to guide Data and Analytics leaders on when to use Event Stream Processing (ESP) platforms instead of alternative tools for real-time processing of streaming data. Pathway is featured as a Unified platform thanks to its "high-performance, Rust-based engine, end-user dashboard builder and in-memory database with streaming ML capabilities".