Pathway Overview
Pathway is a data processing framework which takes care of streaming data updates for you. It makes realtime data processing as easy as it can be.
Installation
You can download the current Pathway release, which is now available in Open Beta on a free-to-use license:
pip install --extra-index-url
[Sign in] to obtain package link
pathway
on a Python 3.8+ installation, and we are ready to roll!
To use Pathway, we only need to import it:
import pathway as pw
Now, you can easily create your processing pipeline while letting Pathway handle the updates! Once your pipeline is created, you can launch the computation on streaming data by a one-line command:
pw.run()
This will launch the computation, each update in the input data streams automatically triggering the update of the whole pipeline! The computation will not stop, waiting for new updates, until the process is killed.
What is Pathway?
You wonder what might be the issues with streaming data and how Pathway resolves them? You can find all the answers and more in our introduction to Pathway.
Your first realtime app with Pathway
To get started with Pathway, follow our tutorial and build a simple your first realtime streaming app using Pathway. You will learn how easy it is to create a end-to-end realtime app using Pathway.
Key concepts
To use Pathway to its full potential, it is primordial to have a basic understanding on the underlying engine of Pathway. Our article on the key concepts of Pathway will provide you valuable insights on how Pathway works.
Streaming and static modes
Pathway is made for streaming data but also provides a static mode to facilitate debugging and testing. All you need to know about those two modes is presented in this article.
Survival guide
Must-read for both first-timers and veterans alike, the survival guide gathers the most commonly used basic elements of Pathway.
To go further
As we continue you will see some more advanced programming constructs which provide a lot of flexibility to Pathway:
- Applying Machine Learning to data tables.
- The ability to do iteration and recursion.
This, and a lot more, is covered in recipes in the Pathway cookbook - try these for a start: