Here, you will get to know the open Pathway framework for data transformation, start to develop code with Pathway, learn to perform rapid prototyping and production deployment, and discover solution templates for various use cases.
In the first place, Pathway was designed to be a life-saver (or at least a time-saver) for Python developers and ML/AI engineers faced with live data sources, where you need to react quickly to fresh data. Still, Pathway is a powerful tool that can be used for a lot of things. If you want to do streaming in Python, build an AI data pipeline, or if you are looking for your next Python data processing framework, keep reading.
Pathway is powered by a scalable Rust engine which is unified across batch and streaming and performs incremental computation. Tasks it can be used for include: Event Stream Processing, real-time Machine Learning, LLM-powered streaming pipelines, Retrieval Augmented Generation and document preparation for LLM, time-series analysis, and graph algorithms.
Pathway allows you to easily develop data transformation pipelines and Machine Learning applications that work with live data sources and changing data. You can start by using a local Python installation, and work in your preferred IDE or notebooks. In this way, you can make data experiments with static data samples and try out data source connections. Once satisfied with the test implementation, the same pipeline can be used in production with live data.
Pathway's engine provides consistent outputs for both batch and streaming data. You can combine real-time and historical data in the same code logic, working with multiple data sources including API's, Kafka-like event brokers, databases, and files.
Start your journey with Pathway with this step-by-step guide, from installation to running your first realtime application.
Learn more about how Pathway's engine and what makes it powerful.Read more
Not sure how to use a specific feature of Pathway? The answer to your question is likely in the API docs.See the API docs
Learn how to use Pathway with our tutorials. For beginners and veterans alike, they cover most of Pathway's features.See the tutorials
Curious about how Pathway works? Don't hesitate to take a look at the sources and clone the repo.Go to Github
Do you want to scale up your Pathway app? Pathway for Enterprise can help you deploy your application on the cloud.Pathway for Enterprise
See the latest available features in the Changelog.See the Changelog
If you've identified a bug, have a feature request, or need technical support, open an issue on GitHub.
Join our community on Discord to connect with other users and get real-time assistance from our team.
For private inquiries or matters that are not suitable for public discussion, feel free to reach out to us.