20k

1 – First Realtime RAG Pipeline

Welcome! In this guide, we’ll walk you through setting up a Realtime RAG app using the GPT-3.5/GPT-4o model (by Open AI) and Pathway. Here, you will create a basic RAG pipeline that uses a set of external files stored in the data folder, extracts relevant information, and updates results as documents change or new ones arrive.

  • You'll use the in-memory vector store/document store by Pathway that is dynamic, fast, scalable and production-ready. While using the same, you won't have to worry about using an external vector database (e.g. Pinecone, Weaviate, etc.).
  • For retrieval: By default, your app uses a local data source to read documents from the data folder. Retrieval is taken care by the Pathway framework hence you don't need to use additional librarires (e.g. FAISS, etc.) for retrievers.
  • Choice of LLM: In your first RAG pipeline you can go for GPT-3.5 as shown ahead. It's a powerful LLM and is one of the cost-effective options provided by the makers of ChatGPT. Alternatively should you wish to use more powerful LLMs from OpenAI such as GPT-4o, o1, etc. – that's doable. But we'll look at them later so it's easy for you to follow a gradual process of hands-on learning.

Key Features

  • Uses an in-memory document store that is easily scalable in enterprise applications.
  • Automatically reacts to the latest changes in your external data store. For example, any change in your Google Drive or Data folder will be reflected in your RAG application right away.
  • The RAG pipeline you build will be production-ready, as you will learn how to deploy it in a containerized environment. This ensures seamless deployment of your Pathway project across various cloud platforms without compatibility issues.

Prerequisites

Before we begin, ensure you have the following prerequisites:

  1. Docker Desktop: This tool allows you to run applications in isolated containers (quick introduction of containerization is below). It ensures consistency across different environments. Download Docker Desktop. (Note: Your antivirus software might block the installation, so temporarily disable it if needed.)
  2. OpenAI API Key: Sign up on the OpenAI website and generate an API key from the API Key Management page. (Remember, don’t share your OpenAI API key with anyone.)

Optional

  • VS Code Docker Extension: If you’re using VS Code, consider installing the Docker extension to manage containers directly from the editor.