Getting Started with Pathway

Pathway is a data processing framework which takes care of streaming data updates for you. It makes real time data processing as easy as it can be.

Installation

You can download the current Pathway release, which is now available in Open Beta on a free-to-use license:

            pip install -U --extra-index-url https://packages.pathway.com/966431ef6ba pathway

on a Python 3.8+ installation, and you are ready to roll!

⚠️ Pathway is available on MacOS and Linux. Pathway is currently not supported on Windows.

You can also try these steps in an online notebook environment like Colab and Noteable.

To use Pathway, you only need to import it:

import pathway as pw

Now, you can easily create your processing pipeline while letting Pathway handle the updates. Once your pipeline is created, you can launch the computation on streaming data by a one-line command:

pw.run()

This will launch the computation. Each update in the input data streams will automatically trigger the update of the whole pipeline. The computation will not stop, and will poll for new updates until the process is killed.

What is Pathway?

Wondering what are the main challenges when working with streaming data and how Pathway resolves them? You can find all the answers and more in our introduction to Pathway.

Your first realtime app with Pathway

To get started with Pathway, follow our tutorial and build your first realtime streaming app using Pathway. You will learn how easy it is to create an end-to-end realtime app using Pathway.

Key concepts

To use Pathway to its full potential, let's get a basic understanding of the underlying engine. To dig deeper, our article on the key concepts of Pathway will provide you valuable insights on how Pathway works.

Streaming and static modes

Pathway is made for streaming data but also provides a static mode to facilitate debugging and testing. All you need to know about those two modes is presented in this article describing the uses of the streaming and static modes of Pathway.

First-steps guide

Must-read for both first-timers and veterans alike, the First-steps guide gathers the most commonly used elements of Pathway.

To go further

As you continue you will see some more advanced programming constructs which provide a lot of flexibility to Pathway, for example:

  • How to apply Machine Learning to data tables?
  • How to do iteration and recursion?

This, and a lot more, is covered in recipes in the Pathway cookbook - try these for a start:

Introduction

What is Pathway?