What is Pathway?

Pathway is a Python-based programming framework for easy building of realtime and reactive data products.

Pathway enables realtime Machine learning with up to 900x smaller latency, taking care of data updates in streaming architectures.

Pathway serves to build more powerful data apps. This means smaller control flow graphs, and cleaner code logic.

Pathway is used by enterprise clients, and the core of the framework is now available on a free-to-use license.

Curious how Pathway resolves key pain points related to streaming data? Our CPO, Adrian, explains this in a quick video introduction to reactive data processing.

(Pathway: Reactive Data Processing in Python, PyData Global, December 2022).

What are the core features of Pathway?

🚀 Realtime Machine Learning

  • Pathway updates your Machine Learning decisions and models automatically, in real time.
  • You can rely on a unified view of all current data.
  • You can treat streaming data just as if they were ordinary tables.

🚀 Reactive design

  • Let your users provide feedback, signal data issues, change settings.
  • Have outputs of your data app which react to live input.
  • Show up-to-date results with sub-second latency.

🚀 Full power of Python

  • Write code in Pathway with all the power and flexibility of Python classes and functions.
  • Interface to the outside world through SQL, Kafka, Debezium, or REST API.

See the full feature list.

What can be built with Pathway

  • Data apps serving responses & insights based on an always up-to-date data model.
  • Data engineering pipelines: Smart-transformation in ELT.
  • Data apps for simulation (what-if scenario testing).
  • Realtime analytics: low-latency analytics on realtime event streaming data.

What data does Pathway work with?

Pathway is specifically designed to store, process, and output any mix of:

  • Pure-SQL data.
  • Time series data.
  • IoT messages (harmonized).
  • Event stream data.
  • Spatiotemporal data (things that move).
  • Graph and ontologies data.

How do I start?

Please start by installing Pathway.

To start writing your own code quickly, take a look at our Pathway survival guide.

You can also take a look at some recipes in the Pathway cookbook:

Building larger applications

First, take a look at the guide to Pathway connectors.

To understand how Pathway can fit into your data processing architecture, take a look at some of the showcases in our pathway-examples repo.