[{"data":1,"prerenderedAt":5552},["ShallowReactive",2],{"\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fllm-app-app":3,"navigation-developers":2052,"\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fllm-app":3991,"surr-\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fllm-app":5549},{"id":4,"title":5,"author":6,"body":7,"description":2036,"extension":2037,"meta":2038,"navigation":405,"path":2047,"seo":2048,"sitemap":2049,"stem":2050,"__hash__":2051},"content\u002F2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F20.llm-app.md","Create your own RAG",null,{"type":8,"value":9,"toc":2018},"minimark",[10,15,19,22,31,39,44,47,114,117,121,124,153,160,167,175,180,187,364,377,416,419,426,437,444,510,517,534,537,563,573,784,787,855,862,866,869,876,930,941,978,985,1067,1070,1073,1107,1110,1188,1205,1209,1212,1237,1244,1251,1287,1290,1311,1314,1317,1321,1324,1327,1333,1432,1443,1451,1636,1640,1655,1724,1731,1735,1750,1870,1874,1877,1894,1898,1901,1907,1923,1926,1938,1941,1966,1969,1973,1976,1979,1982,1986,1989,2011,2014],[11,12,14],"h1",{"id":13},"create-your-own-real-time-rag-with-pathway-live-data-framework","Create your own real-time RAG with Pathway Live Data Framework",[16,17,18],"p",{},"In this guide, you will learn how to construct a dynamic, real-time RAG App using Pathway Live Data Framework and OpenAI.\nRetrieval-Augmented Generation (RAG) is a powerful approach that combines the strengths of information retrieval and generative language models to provide accurate and contextually relevant answers to user queries.",[16,20,21],{},"Indexing the documents in real-time is the only way to make sure the answers you get from your RAG pipeline are based on the latest version of your documents and are not outdated.\nThe Pathway Live Data Framework allows you to easily index in real-time your documents so your RAG is always up-to-date.",[16,23,24,25,30],{},"You can find ready-to-run LLM and RAG examples on our ",[26,27,29],"a",{"href":28},"\u002Fdevelopers\u002Ftemplates?category=llm#llm","App Templates page",".",[32,33],"article-img",{":zoomable":34,"alt":35,"className":36,"src":38},"true","Architecture of the RAG pipeline.",[37],"mx-auto","\u002Fassets\u002Fcontent\u002Fdocumentation\u002FQA-RAG-pipeline.svg",[40,41,43],"h2",{"id":42},"rag-architecture","RAG architecture",[16,45,46],{},"Here's how the RAG structure works:",[48,49,50,61,70,79,88,96,105],"ul",{},[51,52,53,60],"li",{},[26,54,56],{"href":55},"#document-indexing",[57,58,59],"strong",{},"Document Indexing",": The process begins with a collection of documents that are indexed and stored in a searchable format. Indexing involves analyzing the content of each document to identify key terms and phrases, which are then organized for efficient retrieval.",[51,62,63,69],{},[26,64,66],{"href":65},"#user-queries",[57,67,68],{},"User Query",": A user inputs a query, which could be a question or a request for information. This query serves as the starting point for the RAG process.",[51,71,72,78],{},[26,73,75],{"href":74},"#document-retrieval",[57,76,77],{},"Document Retrieval",": The retrieval system takes the user's query and searches through the indexed documents to find the most relevant pieces of information. This step uses advanced algorithms to quickly identify and retrieve documents that are likely to contain the answer to the query.",[51,80,81,87],{},[26,82,84],{"href":83},"#build-the-context-and-the-prompt",[57,85,86],{},"Context Building",": The retrieved documents are then used to build a context. This context includes the relevant information extracted from the documents, which will be used to generate a response.",[51,89,90,95],{},[26,91,92],{"href":83},[57,93,94],{},"Prompt Construction",": A prompt is constructed by combining the user's query with the context built from the retrieved documents. This prompt serves as the input for the generative language model.",[51,97,98,104],{},[26,99,101],{"href":100},"#generate-the-answers-using-an-llm-api",[57,102,103],{},"Answer Generation",": The generative language model processes the prompt and generates a coherent and accurate response. This model is trained to understand and produce human-like text, ensuring that the response is informative and contextually appropriate.",[51,106,107,113],{},[26,108,110],{"href":109},"#returning-answers",[57,111,112],{},"Final Output",": The generated response is presented to the user.",[16,115,116],{},"By integrating retrieval and generation, RAG ensures that the responses are not only accurate but also contextually relevant, making it a robust solution for complex information retrieval tasks.",[40,118,120],{"id":119},"prerequisites","Prerequisites",[16,122,123],{},"Before you start, ensure you have the necessary package installed. You can install it using the following command:",[125,126,131],"pre",{"className":127,"code":128,"language":129,"meta":130,"style":130},"language-bash shiki shiki-themes material-theme-palenight","pip install pathway[xpack-llm] python-dotenv\n","bash","",[132,133,134],"code",{"__ignoreMap":130},[135,136,139,143,147,150],"span",{"class":137,"line":138},"line",1,[135,140,142],{"class":141},"s5Dmg","pip",[135,144,146],{"class":145},"sfyAc"," install",[135,148,149],{"class":145}," pathway[xpack-llm]",[135,151,152],{"class":145}," python-dotenv\n",[16,154,155,156,30],{},"You can learn more about Pathway installation in the ",[26,157,159],{"href":158},"\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Finstallation","relevant article",[16,161,162,163,166],{},"You will also need an OpenAI API key, which should be set into a ",[132,164,165],{},".env"," file:",[125,168,173],{"className":169,"code":171,"language":172},[170],"language-text","OPENAI_API_KEY=\"sk-...\"\n","text",[132,174,171],{"__ignoreMap":130},[176,177,179],"h3",{"id":178},"imports","Imports",[16,181,182,183,186],{},"Import the Pathway and the various components from the ",[132,184,185],{},"xpacks.llm"," package for handling language models.",[125,188,193],{"className":189,"code":190,"filename":191,"language":192,"meta":130,"style":130},"language-python shiki shiki-themes material-theme-palenight","import pathway as pw\nfrom pathway.stdlib.indexing.nearest_neighbors import BruteForceKnnFactory\nfrom pathway.xpacks.llm import llms\nfrom pathway.xpacks.llm.document_store import DocumentStore\nfrom pathway.xpacks.llm.embedders import OpenAIEmbedder\nfrom pathway.xpacks.llm.parsers import UnstructuredParser\nfrom pathway.xpacks.llm.splitters import TokenCountSplitter\n","main.py","python",[132,194,195,211,241,263,289,314,339],{"__ignoreMap":130},[135,196,197,201,205,208],{"class":137,"line":138},[135,198,200],{"class":199},"s6cf3","import",[135,202,204],{"class":203},"s0W1g"," pathway ",[135,206,207],{"class":199},"as",[135,209,210],{"class":203}," pw\n",[135,212,214,217,220,223,226,228,231,233,236,238],{"class":137,"line":213},2,[135,215,216],{"class":199},"from",[135,218,219],{"class":203}," pathway",[135,221,30],{"class":222},"sAklC",[135,224,225],{"class":203},"stdlib",[135,227,30],{"class":222},[135,229,230],{"class":203},"indexing",[135,232,30],{"class":222},[135,234,235],{"class":203},"nearest_neighbors ",[135,237,200],{"class":199},[135,239,240],{"class":203}," BruteForceKnnFactory\n",[135,242,244,246,248,250,253,255,258,260],{"class":137,"line":243},3,[135,245,216],{"class":199},[135,247,219],{"class":203},[135,249,30],{"class":222},[135,251,252],{"class":203},"xpacks",[135,254,30],{"class":222},[135,256,257],{"class":203},"llm ",[135,259,200],{"class":199},[135,261,262],{"class":203}," llms\n",[135,264,266,268,270,272,274,276,279,281,284,286],{"class":137,"line":265},4,[135,267,216],{"class":199},[135,269,219],{"class":203},[135,271,30],{"class":222},[135,273,252],{"class":203},[135,275,30],{"class":222},[135,277,278],{"class":203},"llm",[135,280,30],{"class":222},[135,282,283],{"class":203},"document_store ",[135,285,200],{"class":199},[135,287,288],{"class":203}," DocumentStore\n",[135,290,292,294,296,298,300,302,304,306,309,311],{"class":137,"line":291},5,[135,293,216],{"class":199},[135,295,219],{"class":203},[135,297,30],{"class":222},[135,299,252],{"class":203},[135,301,30],{"class":222},[135,303,278],{"class":203},[135,305,30],{"class":222},[135,307,308],{"class":203},"embedders ",[135,310,200],{"class":199},[135,312,313],{"class":203}," OpenAIEmbedder\n",[135,315,317,319,321,323,325,327,329,331,334,336],{"class":137,"line":316},6,[135,318,216],{"class":199},[135,320,219],{"class":203},[135,322,30],{"class":222},[135,324,252],{"class":203},[135,326,30],{"class":222},[135,328,278],{"class":203},[135,330,30],{"class":222},[135,332,333],{"class":203},"parsers ",[135,335,200],{"class":199},[135,337,338],{"class":203}," UnstructuredParser\n",[135,340,342,344,346,348,350,352,354,356,359,361],{"class":137,"line":341},7,[135,343,216],{"class":199},[135,345,219],{"class":203},[135,347,30],{"class":222},[135,349,252],{"class":203},[135,351,30],{"class":222},[135,353,278],{"class":203},[135,355,30],{"class":222},[135,357,358],{"class":203},"splitters ",[135,360,200],{"class":199},[135,362,363],{"class":203}," TokenCountSplitter\n",[16,365,366,367,370,371,374,375,166],{},"You will also need ",[132,368,369],{},"os"," and ",[132,372,373],{},"dotenv"," to load the API key from the ",[132,376,165],{},[125,378,380],{"className":189,"code":379,"filename":191,"language":192,"meta":130,"style":130},"from dotenv import load_dotenv\nimport os\n\nload_dotenv()\n",[132,381,382,394,401,407],{"__ignoreMap":130},[135,383,384,386,389,391],{"class":137,"line":138},[135,385,216],{"class":199},[135,387,388],{"class":203}," dotenv ",[135,390,200],{"class":199},[135,392,393],{"class":203}," load_dotenv\n",[135,395,396,398],{"class":137,"line":213},[135,397,200],{"class":199},[135,399,400],{"class":203}," os\n",[135,402,403],{"class":137,"line":243},[135,404,406],{"emptyLinePlaceholder":405},true,"\n",[135,408,409,413],{"class":137,"line":265},[135,410,412],{"class":411},"sdLwU","load_dotenv",[135,414,415],{"class":222},"()\n",[40,417,59],{"id":418},"document-indexing",[16,420,421,422,30],{},"The first step is to index your documents.\nYou need to ingest them into the pipeline using ",[26,423,425],{"href":424},"\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors-overview","Pathway Live Data Framework connectors",[16,427,428,429,432,433,436],{},"This example will focus on ",[57,430,431],{},"PDF documents stored locally on the file system",", in a ",[132,434,435],{},".\u002Fdata\u002F"," directory.",[16,438,439,440,30],{},"You can load the documents from a specified directory using the ",[26,441,443],{"href":442},"\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Ffs-connector","file system connector",[125,445,447],{"className":189,"code":446,"filename":191,"language":192,"meta":130,"style":130},"documents = pw.io.fs.read(\".\u002Fdata\u002F\", format=\"binary\", with_metadata=True)\n",[132,448,449],{"__ignoreMap":130},[135,450,451,454,457,460,462,466,468,471,473,476,479,482,484,486,489,493,495,497,500,502,504,507],{"class":137,"line":138},[135,452,453],{"class":203},"documents ",[135,455,456],{"class":222},"=",[135,458,459],{"class":203}," pw",[135,461,30],{"class":222},[135,463,465],{"class":464},"s-wAU","io",[135,467,30],{"class":222},[135,469,470],{"class":464},"fs",[135,472,30],{"class":222},[135,474,475],{"class":411},"read",[135,477,478],{"class":222},"(",[135,480,481],{"class":222},"\"",[135,483,435],{"class":145},[135,485,481],{"class":222},[135,487,488],{"class":222},",",[135,490,492],{"class":491},"s7ZW3"," format",[135,494,456],{"class":222},[135,496,481],{"class":222},[135,498,499],{"class":145},"binary",[135,501,481],{"class":222},[135,503,488],{"class":222},[135,505,506],{"class":491}," with_metadata",[135,508,509],{"class":222},"=True)\n",[16,511,512,513,30],{},"You can find the list of Pathway Live Data Framework connectors ",[26,514,516],{"href":515},"\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Flive-data-framework-connectors","here",[16,518,519,520,524,525,527,528,30],{},"Now, you need to index those documents.\nThis can be easily done in Pathway Live Data Framework using a ",[26,521,523],{"href":522},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fdocument_store","DocumentStore"," which stores and manages the documents and their embeddings.\nThe ",[132,526,523],{}," handles the parsing, post-processing and splitting. It then indexes the documents and provide a retrieval function ",[26,529,531],{"href":530},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fdocument_store#pathway.xpacks.llm.document_store.DocumentStore.retrieve_query",[132,532,533],{},"retrieve_query",[16,535,536],{},"The document store requires the following parameters:",[48,538,539,545,551,557],{},[51,540,541,544],{},[57,542,543],{},"Text Splitter",": Splits documents into manageable chunks based on token count.",[51,546,547,550],{},[57,548,549],{},"Embedder",": Converts text into embeddings, which are numerical representations that capture semantic meaning.",[51,552,553,556],{},[57,554,555],{},"Retriever Factory",": Uses embeddings to find the most relevant documents for a given query.",[51,558,559,562],{},[57,560,561],{},"Parser",": Extracts and structures text from documents.",[16,564,565,566,572],{},"In this example, you will do a token-based chunking compatible with OpenAI's embedding models.\nYou can use the ",[26,567,569],{"href":568},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fembedders#pathway.xpacks.llm.embedders.OpenAIEmbedder",[132,570,571],{},"OpenAIEmbedder"," to use OpenAI Embedding services.\nFor the retrieval of the documents, let's use a simple brute force strategy.",[125,574,576],{"className":189,"code":575,"filename":191,"language":192,"meta":130,"style":130},"text_splitter = TokenCountSplitter(\n    min_tokens=100, max_tokens=500, encoding_name=\"cl100k_base\"\n)\nembedder = OpenAIEmbedder(api_key=os.environ[\"OPENAI_API_KEY\"])\nretriever_factory = BruteForceKnnFactory(\n    embedder=embedder,\n)\nparser = UnstructuredParser(\n    chunking_mode=\"by_title\",\n    chunking_kwargs={\n        \"max_characters\": 3000,\n        \"new_after_n_chars\": 2000,\n    },\n)\n",[132,577,578,591,627,632,669,681,694,698,711,728,737,756,773,779],{"__ignoreMap":130},[135,579,580,583,585,588],{"class":137,"line":138},[135,581,582],{"class":203},"text_splitter ",[135,584,456],{"class":222},[135,586,587],{"class":411}," TokenCountSplitter",[135,589,590],{"class":222},"(\n",[135,592,593,596,598,602,604,607,609,612,614,617,619,621,624],{"class":137,"line":213},[135,594,595],{"class":491},"    min_tokens",[135,597,456],{"class":222},[135,599,601],{"class":600},"sx098","100",[135,603,488],{"class":222},[135,605,606],{"class":491}," max_tokens",[135,608,456],{"class":222},[135,610,611],{"class":600},"500",[135,613,488],{"class":222},[135,615,616],{"class":491}," encoding_name",[135,618,456],{"class":222},[135,620,481],{"class":222},[135,622,623],{"class":145},"cl100k_base",[135,625,626],{"class":222},"\"\n",[135,628,629],{"class":137,"line":243},[135,630,631],{"class":222},")\n",[135,633,634,637,639,642,644,647,649,651,653,656,659,661,664,666],{"class":137,"line":265},[135,635,636],{"class":203},"embedder ",[135,638,456],{"class":222},[135,640,641],{"class":411}," OpenAIEmbedder",[135,643,478],{"class":222},[135,645,646],{"class":491},"api_key",[135,648,456],{"class":222},[135,650,369],{"class":411},[135,652,30],{"class":222},[135,654,655],{"class":464},"environ",[135,657,658],{"class":222},"[",[135,660,481],{"class":222},[135,662,663],{"class":145},"OPENAI_API_KEY",[135,665,481],{"class":222},[135,667,668],{"class":222},"])\n",[135,670,671,674,676,679],{"class":137,"line":291},[135,672,673],{"class":203},"retriever_factory ",[135,675,456],{"class":222},[135,677,678],{"class":411}," BruteForceKnnFactory",[135,680,590],{"class":222},[135,682,683,686,688,691],{"class":137,"line":316},[135,684,685],{"class":491},"    embedder",[135,687,456],{"class":222},[135,689,690],{"class":411},"embedder",[135,692,693],{"class":222},",\n",[135,695,696],{"class":137,"line":341},[135,697,631],{"class":222},[135,699,701,704,706,709],{"class":137,"line":700},8,[135,702,703],{"class":203},"parser ",[135,705,456],{"class":222},[135,707,708],{"class":411}," UnstructuredParser",[135,710,590],{"class":222},[135,712,714,717,719,721,724,726],{"class":137,"line":713},9,[135,715,716],{"class":491},"    chunking_mode",[135,718,456],{"class":222},[135,720,481],{"class":222},[135,722,723],{"class":145},"by_title",[135,725,481],{"class":222},[135,727,693],{"class":222},[135,729,731,734],{"class":137,"line":730},10,[135,732,733],{"class":491},"    chunking_kwargs",[135,735,736],{"class":222},"={\n",[135,738,740,743,746,748,751,754],{"class":137,"line":739},11,[135,741,742],{"class":222},"        \"",[135,744,745],{"class":145},"max_characters",[135,747,481],{"class":222},[135,749,750],{"class":222},":",[135,752,753],{"class":600}," 3000",[135,755,693],{"class":222},[135,757,759,761,764,766,768,771],{"class":137,"line":758},12,[135,760,742],{"class":222},[135,762,763],{"class":145},"new_after_n_chars",[135,765,481],{"class":222},[135,767,750],{"class":222},[135,769,770],{"class":600}," 2000",[135,772,693],{"class":222},[135,774,776],{"class":137,"line":775},13,[135,777,778],{"class":222},"    },\n",[135,780,782],{"class":137,"line":781},14,[135,783,631],{"class":222},[16,785,786],{},"With all components ready, you can create the document store:",[125,788,790],{"className":189,"code":789,"filename":191,"language":192,"meta":130,"style":130},"document_store = DocumentStore(\n    docs=documents,\n    retriever_factory=retriever_factory,\n    parser=parser,\n    splitter=text_splitter,\n)\n",[132,791,792,803,815,827,839,851],{"__ignoreMap":130},[135,793,794,796,798,801],{"class":137,"line":138},[135,795,283],{"class":203},[135,797,456],{"class":222},[135,799,800],{"class":411}," DocumentStore",[135,802,590],{"class":222},[135,804,805,808,810,813],{"class":137,"line":213},[135,806,807],{"class":491},"    docs",[135,809,456],{"class":222},[135,811,812],{"class":411},"documents",[135,814,693],{"class":222},[135,816,817,820,822,825],{"class":137,"line":243},[135,818,819],{"class":491},"    retriever_factory",[135,821,456],{"class":222},[135,823,824],{"class":411},"retriever_factory",[135,826,693],{"class":222},[135,828,829,832,834,837],{"class":137,"line":265},[135,830,831],{"class":491},"    parser",[135,833,456],{"class":222},[135,835,836],{"class":411},"parser",[135,838,693],{"class":222},[135,840,841,844,846,849],{"class":137,"line":291},[135,842,843],{"class":491},"    splitter",[135,845,456],{"class":222},[135,847,848],{"class":411},"text_splitter",[135,850,693],{"class":222},[135,852,853],{"class":137,"line":316},[135,854,631],{"class":222},[16,856,857,858,30],{},"You can learn more about document indexing in Pathway Live Data Framework in our ",[26,859,861],{"href":860},"\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fdocs-indexing","dedicated article",[40,863,865],{"id":864},"user-queries","User queries",[16,867,868],{},"To receive the users' queries and forward the answers, let's use a lightweight HTTP server.",[16,870,871,872,30],{},"You can configure the HTTP server using the ",[26,873,875],{"href":874},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fhttp#pathway.io.http.PathwayWebserver","PathwayWebserver class",[125,877,879],{"className":189,"code":878,"language":192,"meta":130,"style":130},"webserver = pw.io.http.PathwayWebserver(host=\"0.0.0.0\", port=8011)\n",[132,880,881],{"__ignoreMap":130},[135,882,883,886,888,890,892,894,896,899,901,904,906,909,911,913,916,918,920,923,925,928],{"class":137,"line":138},[135,884,885],{"class":203},"webserver ",[135,887,456],{"class":222},[135,889,459],{"class":203},[135,891,30],{"class":222},[135,893,465],{"class":464},[135,895,30],{"class":222},[135,897,898],{"class":464},"http",[135,900,30],{"class":222},[135,902,903],{"class":411},"PathwayWebserver",[135,905,478],{"class":222},[135,907,908],{"class":491},"host",[135,910,456],{"class":222},[135,912,481],{"class":222},[135,914,915],{"class":145},"0.0.0.0",[135,917,481],{"class":222},[135,919,488],{"class":222},[135,921,922],{"class":491}," port",[135,924,456],{"class":222},[135,926,927],{"class":600},"8011",[135,929,631],{"class":222},[16,931,932,933,937,938,30],{},"You also need to define the input schema, using ",[26,934,936],{"href":935},"\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fschema","pw.Schema",", which helps to enforce the structure of the data being processed by Pathway Live Data Framework.\nIn our example, the queries will have a single field ",[132,939,940],{},"messages",[125,942,944],{"className":189,"code":943,"language":192,"meta":130,"style":130},"class QuerySchema(pw.Schema):\n    messages: str\n",[132,945,946,968],{"__ignoreMap":130},[135,947,948,952,955,957,960,962,965],{"class":137,"line":138},[135,949,951],{"class":950},"sJ14y","class",[135,953,954],{"class":141}," QuerySchema",[135,956,478],{"class":222},[135,958,959],{"class":141},"pw",[135,961,30],{"class":222},[135,963,964],{"class":141},"Schema",[135,966,967],{"class":222},"):\n",[135,969,970,973,975],{"class":137,"line":213},[135,971,972],{"class":203},"    messages",[135,974,750],{"class":222},[135,976,977],{"class":141}," str\n",[16,979,980,981,30],{},"Once everything is ready, you can create the server using the ",[26,982,984],{"href":983},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fhttp#pathway.io.http.rest_connector","REST connector",[125,986,988],{"className":189,"code":987,"filename":191,"language":192,"meta":130,"style":130},"queries, writer = pw.io.http.rest_connector(\n    webserver=webserver,\n    schema=QuerySchema,\n    autocommit_duration_ms=50,\n    delete_completed_queries=False,\n)\n",[132,989,990,1019,1031,1043,1055,1063],{"__ignoreMap":130},[135,991,992,995,997,1000,1002,1004,1006,1008,1010,1012,1014,1017],{"class":137,"line":138},[135,993,994],{"class":203},"queries",[135,996,488],{"class":222},[135,998,999],{"class":203}," writer ",[135,1001,456],{"class":222},[135,1003,459],{"class":203},[135,1005,30],{"class":222},[135,1007,465],{"class":464},[135,1009,30],{"class":222},[135,1011,898],{"class":464},[135,1013,30],{"class":222},[135,1015,1016],{"class":411},"rest_connector",[135,1018,590],{"class":222},[135,1020,1021,1024,1026,1029],{"class":137,"line":213},[135,1022,1023],{"class":491},"    webserver",[135,1025,456],{"class":222},[135,1027,1028],{"class":411},"webserver",[135,1030,693],{"class":222},[135,1032,1033,1036,1038,1041],{"class":137,"line":243},[135,1034,1035],{"class":491},"    schema",[135,1037,456],{"class":222},[135,1039,1040],{"class":411},"QuerySchema",[135,1042,693],{"class":222},[135,1044,1045,1048,1050,1053],{"class":137,"line":265},[135,1046,1047],{"class":491},"    autocommit_duration_ms",[135,1049,456],{"class":222},[135,1051,1052],{"class":600},"50",[135,1054,693],{"class":222},[135,1056,1057,1060],{"class":137,"line":291},[135,1058,1059],{"class":491},"    delete_completed_queries",[135,1061,1062],{"class":222},"=False,\n",[135,1064,1065],{"class":137,"line":316},[135,1066,631],{"class":222},[16,1068,1069],{},"Now users can send queries to Pathway Live Data Framework via HTTP requests.",[16,1071,1072],{},"For document retrieval, you need to format the queries to fit the schema expected by the document store:",[48,1074,1075,1083,1091,1099],{},[51,1076,1077,1082],{},[57,1078,1079],{},[132,1080,1081],{},"query",": the query sent by the user,",[51,1084,1085,1090],{},[57,1086,1087],{},[132,1088,1089],{},"k",": the number of documents to retrieve for the query,",[51,1092,1093,1098],{},[57,1094,1095],{},[132,1096,1097],{},"metadata_filter"," (optional): filters using metadata,",[51,1100,1101,1106],{},[57,1102,1103],{},[132,1104,1105],{},"filepath_globalpattern"," (optional): path pattern used to filter files.",[16,1108,1109],{},"Let's only retrieve a single document per query, with no filtering:",[125,1111,1113],{"className":189,"code":1112,"filename":191,"language":192,"meta":130,"style":130},"queries = queries.select(\n    query = pw.this.messages,\n    k = 1,\n    metadata_filter = None,\n    filepath_globpattern = None,\n)\n",[132,1114,1115,1132,1153,1165,1175,1184],{"__ignoreMap":130},[135,1116,1117,1120,1122,1125,1127,1130],{"class":137,"line":138},[135,1118,1119],{"class":203},"queries ",[135,1121,456],{"class":222},[135,1123,1124],{"class":203}," queries",[135,1126,30],{"class":222},[135,1128,1129],{"class":411},"select",[135,1131,590],{"class":222},[135,1133,1134,1137,1140,1142,1144,1147,1149,1151],{"class":137,"line":213},[135,1135,1136],{"class":491},"    query",[135,1138,1139],{"class":222}," =",[135,1141,459],{"class":411},[135,1143,30],{"class":222},[135,1145,1146],{"class":464},"this",[135,1148,30],{"class":222},[135,1150,940],{"class":464},[135,1152,693],{"class":222},[135,1154,1155,1158,1160,1163],{"class":137,"line":243},[135,1156,1157],{"class":491},"    k",[135,1159,1139],{"class":222},[135,1161,1162],{"class":600}," 1",[135,1164,693],{"class":222},[135,1166,1167,1170,1172],{"class":137,"line":265},[135,1168,1169],{"class":491},"    metadata_filter",[135,1171,1139],{"class":222},[135,1173,1174],{"class":222}," None,\n",[135,1176,1177,1180,1182],{"class":137,"line":291},[135,1178,1179],{"class":491},"    filepath_globpattern",[135,1181,1139],{"class":222},[135,1183,1174],{"class":222},[135,1185,1186],{"class":137,"line":316},[135,1187,631],{"class":222},[16,1189,1190,1191,1194,1195,1199,1200,1202,1203,30],{},"Note that here we use a single value (",[132,1192,1193],{},"k=1",") for ",[1196,1197,1198],"em",{},"all"," the queries, but this is something that can be adapted per query.\nFor example, this can be specified by the used in the query with an additional parameter ",[132,1201,1089],{}," in ",[132,1204,1040],{},[40,1206,1208],{"id":1207},"document-retrieval","Document retrieval",[16,1210,1211],{},"Now that we have our index and our queries, you need to retrieve the most relevant documents for each query.\nThis is easily done using the document store:",[125,1213,1215],{"className":189,"code":1214,"filename":191,"language":192,"meta":130,"style":130},"retrieved_documents = document_store.retrieve_query(queries)\n",[132,1216,1217],{"__ignoreMap":130},[135,1218,1219,1222,1224,1227,1229,1231,1233,1235],{"class":137,"line":138},[135,1220,1221],{"class":203},"retrieved_documents ",[135,1223,456],{"class":222},[135,1225,1226],{"class":203}," document_store",[135,1228,30],{"class":222},[135,1230,533],{"class":411},[135,1232,478],{"class":222},[135,1234,994],{"class":411},[135,1236,631],{"class":222},[16,1238,1239,1240,1243],{},"The retrieved documents are stored in the column ",[132,1241,1242],{},"result",".\nThis column contains a JSON containing both the content and other metadata.",[16,1245,1246,1247,1250],{},"Let's rename this column ",[132,1248,1249],{},"docs"," for clarity reasons.",[125,1252,1254],{"className":189,"code":1253,"filename":191,"language":192,"meta":130,"style":130},"retrieved_documents = retrieved_documents.select(docs=pw.this.result)\n",[132,1255,1256],{"__ignoreMap":130},[135,1257,1258,1260,1262,1265,1267,1269,1271,1273,1275,1277,1279,1281,1283,1285],{"class":137,"line":138},[135,1259,1221],{"class":203},[135,1261,456],{"class":222},[135,1263,1264],{"class":203}," retrieved_documents",[135,1266,30],{"class":222},[135,1268,1129],{"class":411},[135,1270,478],{"class":222},[135,1272,1249],{"class":491},[135,1274,456],{"class":222},[135,1276,959],{"class":411},[135,1278,30],{"class":222},[135,1280,1146],{"class":464},[135,1282,30],{"class":222},[135,1284,1242],{"class":464},[135,1286,631],{"class":222},[16,1288,1289],{},"Combine the original queries with the retrieved documents to create a context for generating answers.",[125,1291,1293],{"className":189,"code":1292,"filename":191,"language":192,"meta":130,"style":130},"queries_context = queries + retrieved_documents\n",[132,1294,1295],{"__ignoreMap":130},[135,1296,1297,1300,1302,1305,1308],{"class":137,"line":138},[135,1298,1299],{"class":203},"queries_context ",[135,1301,456],{"class":222},[135,1303,1304],{"class":203}," queries ",[135,1306,1307],{"class":222},"+",[135,1309,1310],{"class":203}," retrieved_documents\n",[40,1312,103],{"id":1313},"answer-generation",[16,1315,1316],{},"Using the query and its associated documents, you can build the prompt and generate an answer using an LLM.",[176,1318,1320],{"id":1319},"build-the-context-and-the-prompt","Build the context and the prompt",[16,1322,1323],{},"The context is the information the LLM needs to answer the question: it's the content of the retrieved documents.",[16,1325,1326],{},"Create a context from the retrieved documents and build a prompt that includes both the context and the user's query.\nThis prompt will be used to generate the final answer.",[16,1328,1329,1330,1332],{},"The context will simply be the content of the documents put one after the other.\nThe content is stored in the ",[132,1331,172],{}," part of the JSON.",[125,1334,1336],{"className":189,"code":1335,"filename":191,"language":192,"meta":130,"style":130},"def get_context(documents):\n    content_list = []\n    for doc in documents:\n        content_list.append(str(doc[\"text\"]))\n    return \" \".join(content_list)\n",[132,1337,1338,1352,1362,1379,1410],{"__ignoreMap":130},[135,1339,1340,1343,1346,1348,1350],{"class":137,"line":138},[135,1341,1342],{"class":950},"def",[135,1344,1345],{"class":411}," get_context",[135,1347,478],{"class":222},[135,1349,812],{"class":491},[135,1351,967],{"class":222},[135,1353,1354,1357,1359],{"class":137,"line":213},[135,1355,1356],{"class":203},"    content_list ",[135,1358,456],{"class":222},[135,1360,1361],{"class":222}," []\n",[135,1363,1364,1367,1370,1373,1376],{"class":137,"line":243},[135,1365,1366],{"class":199},"    for",[135,1368,1369],{"class":203}," doc ",[135,1371,1372],{"class":199},"in",[135,1374,1375],{"class":203}," documents",[135,1377,1378],{"class":222},":\n",[135,1380,1381,1384,1386,1389,1391,1394,1396,1399,1401,1403,1405,1407],{"class":137,"line":265},[135,1382,1383],{"class":203},"        content_list",[135,1385,30],{"class":222},[135,1387,1388],{"class":411},"append",[135,1390,478],{"class":222},[135,1392,1393],{"class":141},"str",[135,1395,478],{"class":222},[135,1397,1398],{"class":411},"doc",[135,1400,658],{"class":222},[135,1402,481],{"class":222},[135,1404,172],{"class":145},[135,1406,481],{"class":222},[135,1408,1409],{"class":222},"]))\n",[135,1411,1412,1415,1418,1420,1422,1425,1427,1430],{"class":137,"line":291},[135,1413,1414],{"class":199},"    return",[135,1416,1417],{"class":222}," \"",[135,1419,1417],{"class":222},[135,1421,30],{"class":222},[135,1423,1424],{"class":411},"join",[135,1426,478],{"class":222},[135,1428,1429],{"class":411},"content_list",[135,1431,631],{"class":222},[1433,1434,1435],"note",{},[16,1436,1437,1440,1441,30],{},[57,1438,1439],{},"Note",": this function will be applied on a single row, not a table.\nAlso, in our example there is no need to store all the content in a list since there is only a single retrieved document.\nThis was done to be sure it works with higher values of ",[132,1442,1089],{},[16,1444,1445,1446,1450],{},"You can build a ",[26,1447,1449],{"href":1448},"\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Fuser-defined-functions","UDF"," to create the prompt:",[125,1452,1454],{"className":189,"code":1453,"filename":191,"language":192,"meta":130,"style":130},"@pw.udf\ndef build_prompts_udf(documents, query) -> str:\n    context = get_context(documents)\n    prompt = (\n        f\"Given the following documents : \\n {context} \\nanswer this query: {query}\"\n    )\n    return prompt\n\n\nprompts = queries_context+queries_context.select(\n    prompts=build_prompts_udf(pw.this.docs, pw.this.query)\n)\n",[132,1455,1456,1468,1495,1510,1520,1555,1560,1567,1571,1575,1596,1632],{"__ignoreMap":130},[135,1457,1458,1461,1463,1465],{"class":137,"line":138},[135,1459,1460],{"class":222},"@",[135,1462,959],{"class":411},[135,1464,30],{"class":222},[135,1466,1467],{"class":411},"udf\n",[135,1469,1470,1472,1475,1477,1479,1481,1484,1487,1490,1493],{"class":137,"line":213},[135,1471,1342],{"class":950},[135,1473,1474],{"class":411}," build_prompts_udf",[135,1476,478],{"class":222},[135,1478,812],{"class":491},[135,1480,488],{"class":222},[135,1482,1483],{"class":491}," query",[135,1485,1486],{"class":222},")",[135,1488,1489],{"class":222}," ->",[135,1491,1492],{"class":141}," str",[135,1494,1378],{"class":222},[135,1496,1497,1500,1502,1504,1506,1508],{"class":137,"line":243},[135,1498,1499],{"class":203},"    context ",[135,1501,456],{"class":222},[135,1503,1345],{"class":411},[135,1505,478],{"class":222},[135,1507,812],{"class":411},[135,1509,631],{"class":222},[135,1511,1512,1515,1517],{"class":137,"line":265},[135,1513,1514],{"class":203},"    prompt ",[135,1516,456],{"class":222},[135,1518,1519],{"class":222}," (\n",[135,1521,1522,1525,1528,1531,1534,1537,1540,1543,1546,1549,1551,1553],{"class":137,"line":291},[135,1523,1524],{"class":950},"        f",[135,1526,1527],{"class":145},"\"Given the following documents : ",[135,1529,1530],{"class":203},"\\n",[135,1532,1533],{"class":600}," {",[135,1535,1536],{"class":203},"context",[135,1538,1539],{"class":600},"}",[135,1541,1542],{"class":203}," \\n",[135,1544,1545],{"class":145},"answer this query: ",[135,1547,1548],{"class":600},"{",[135,1550,1081],{"class":203},[135,1552,1539],{"class":600},[135,1554,626],{"class":145},[135,1556,1557],{"class":137,"line":316},[135,1558,1559],{"class":222},"    )\n",[135,1561,1562,1564],{"class":137,"line":341},[135,1563,1414],{"class":199},[135,1565,1566],{"class":203}," prompt\n",[135,1568,1569],{"class":137,"line":700},[135,1570,406],{"emptyLinePlaceholder":405},[135,1572,1573],{"class":137,"line":713},[135,1574,406],{"emptyLinePlaceholder":405},[135,1576,1577,1580,1582,1585,1587,1590,1592,1594],{"class":137,"line":730},[135,1578,1579],{"class":203},"prompts ",[135,1581,456],{"class":222},[135,1583,1584],{"class":203}," queries_context",[135,1586,1307],{"class":222},[135,1588,1589],{"class":203},"queries_context",[135,1591,30],{"class":222},[135,1593,1129],{"class":411},[135,1595,590],{"class":222},[135,1597,1598,1601,1603,1606,1608,1610,1612,1614,1616,1618,1620,1622,1624,1626,1628,1630],{"class":137,"line":739},[135,1599,1600],{"class":491},"    prompts",[135,1602,456],{"class":222},[135,1604,1605],{"class":411},"build_prompts_udf",[135,1607,478],{"class":222},[135,1609,959],{"class":411},[135,1611,30],{"class":222},[135,1613,1146],{"class":464},[135,1615,30],{"class":222},[135,1617,1249],{"class":464},[135,1619,488],{"class":222},[135,1621,459],{"class":411},[135,1623,30],{"class":222},[135,1625,1146],{"class":464},[135,1627,30],{"class":222},[135,1629,1081],{"class":464},[135,1631,631],{"class":222},[135,1633,1634],{"class":137,"line":758},[135,1635,631],{"class":222},[176,1637,1639],{"id":1638},"define-the-model","Define the model",[16,1641,1642,1643,1647,1648,1654],{},"The Pathway Live Data Framework provides ",[26,1644,1646],{"href":1645},"\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fllm-chats","wrappers"," for most of the LLM providers.\nYou will use the ",[26,1649,1651],{"href":1650},"\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fllm-chats#openaichat",[132,1652,1653],{},"OpenAIChat"," to send prompt to OpenAI.\nYou need to specify the language model and your API key:",[125,1656,1658],{"className":189,"code":1657,"filename":191,"language":192,"meta":130,"style":130},"model = llms.OpenAIChat(\n    model=\"gpt-4o-mini\",\n    api_key=os.environ[\"OPENAI_API_KEY\"], # Read OpenAI API key from environmental variables\n)\n",[132,1659,1660,1676,1692,1720],{"__ignoreMap":130},[135,1661,1662,1665,1667,1670,1672,1674],{"class":137,"line":138},[135,1663,1664],{"class":203},"model ",[135,1666,456],{"class":222},[135,1668,1669],{"class":203}," llms",[135,1671,30],{"class":222},[135,1673,1653],{"class":411},[135,1675,590],{"class":222},[135,1677,1678,1681,1683,1685,1688,1690],{"class":137,"line":213},[135,1679,1680],{"class":491},"    model",[135,1682,456],{"class":222},[135,1684,481],{"class":222},[135,1686,1687],{"class":145},"gpt-4o-mini",[135,1689,481],{"class":222},[135,1691,693],{"class":222},[135,1693,1694,1697,1699,1701,1703,1705,1707,1709,1711,1713,1716],{"class":137,"line":243},[135,1695,1696],{"class":491},"    api_key",[135,1698,456],{"class":222},[135,1700,369],{"class":411},[135,1702,30],{"class":222},[135,1704,655],{"class":464},[135,1706,658],{"class":222},[135,1708,481],{"class":222},[135,1710,663],{"class":145},[135,1712,481],{"class":222},[135,1714,1715],{"class":222},"],",[135,1717,1719],{"class":1718},"saEQR"," # Read OpenAI API key from environmental variables\n",[135,1721,1722],{"class":137,"line":265},[135,1723,631],{"class":222},[16,1725,1726,1727,1730],{},"This ",[132,1728,1729],{},"model"," is a UDF and can be directly applied to a column.",[176,1732,1734],{"id":1733},"generate-the-answers-using-an-llm-api","Generate the answers using an LLM API",[16,1736,1737,1738,1740,1741,1747,1748,30],{},"Using ",[132,1739,1653],{},", you can ask OpenAI to answer your prompts using the wanted LLM.\nUse the ",[26,1742,1744],{"href":1743},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fllms#pathway.xpacks.llm.llms.prompt_chat_single_qa",[132,1745,1746],{},"prompt_chat_single_qa"," to transforms the questions into prompts compatible with ",[132,1749,1729],{},[125,1751,1753],{"className":189,"code":1752,"filename":191,"language":192,"meta":130,"style":130},"response = prompts.select(\n    *pw.this.without(pw.this.query, pw.this.prompts, pw.this.docs),\n    result=model(\n        llms.prompt_chat_single_qa(pw.this.prompts),\n    ),\n)\n",[132,1754,1755,1771,1827,1838,1861,1866],{"__ignoreMap":130},[135,1756,1757,1760,1762,1765,1767,1769],{"class":137,"line":138},[135,1758,1759],{"class":203},"response ",[135,1761,456],{"class":222},[135,1763,1764],{"class":203}," prompts",[135,1766,30],{"class":222},[135,1768,1129],{"class":411},[135,1770,590],{"class":222},[135,1772,1773,1776,1778,1780,1782,1784,1787,1789,1791,1793,1795,1797,1799,1801,1803,1805,1807,1809,1812,1814,1816,1818,1820,1822,1824],{"class":137,"line":213},[135,1774,1775],{"class":222},"    *",[135,1777,959],{"class":411},[135,1779,30],{"class":222},[135,1781,1146],{"class":464},[135,1783,30],{"class":222},[135,1785,1786],{"class":411},"without",[135,1788,478],{"class":222},[135,1790,959],{"class":411},[135,1792,30],{"class":222},[135,1794,1146],{"class":464},[135,1796,30],{"class":222},[135,1798,1081],{"class":464},[135,1800,488],{"class":222},[135,1802,459],{"class":411},[135,1804,30],{"class":222},[135,1806,1146],{"class":464},[135,1808,30],{"class":222},[135,1810,1811],{"class":464},"prompts",[135,1813,488],{"class":222},[135,1815,459],{"class":411},[135,1817,30],{"class":222},[135,1819,1146],{"class":464},[135,1821,30],{"class":222},[135,1823,1249],{"class":464},[135,1825,1826],{"class":222},"),\n",[135,1828,1829,1832,1834,1836],{"class":137,"line":243},[135,1830,1831],{"class":491},"    result",[135,1833,456],{"class":222},[135,1835,1729],{"class":411},[135,1837,590],{"class":222},[135,1839,1840,1843,1845,1847,1849,1851,1853,1855,1857,1859],{"class":137,"line":265},[135,1841,1842],{"class":411},"        llms",[135,1844,30],{"class":222},[135,1846,1746],{"class":411},[135,1848,478],{"class":222},[135,1850,959],{"class":411},[135,1852,30],{"class":222},[135,1854,1146],{"class":464},[135,1856,30],{"class":222},[135,1858,1811],{"class":464},[135,1860,1826],{"class":222},[135,1862,1863],{"class":137,"line":291},[135,1864,1865],{"class":222},"    ),\n",[135,1867,1868],{"class":137,"line":316},[135,1869,631],{"class":222},[40,1871,1873],{"id":1872},"returning-answers","Returning Answers",[16,1875,1876],{},"Send the generated answers back to the user via the HTTP server using the writer.",[125,1878,1880],{"className":189,"code":1879,"filename":191,"language":192,"meta":130,"style":130},"writer(responses)\n",[132,1881,1882],{"__ignoreMap":130},[135,1883,1884,1887,1889,1892],{"class":137,"line":138},[135,1885,1886],{"class":411},"writer",[135,1888,478],{"class":222},[135,1890,1891],{"class":411},"responses",[135,1893,631],{"class":222},[40,1895,1897],{"id":1896},"run-the-pipeline-and-try-it-out","Run the pipeline and try it out!",[16,1899,1900],{},"That's it!",[16,1902,1903,1904],{},"Now that your pipeline is ready, you need to run it using ",[132,1905,1906],{},"pw.run()",[125,1908,1910],{"className":189,"code":1909,"filename":191,"language":192,"meta":130,"style":130},"pw.run()\n",[132,1911,1912],{"__ignoreMap":130},[135,1913,1914,1916,1918,1921],{"class":137,"line":138},[135,1915,959],{"class":203},[135,1917,30],{"class":222},[135,1919,1920],{"class":411},"run",[135,1922,415],{"class":222},[16,1924,1925],{},"Start the Python script to launch the web server and begin processing queries.",[125,1927,1929],{"className":127,"code":1928,"language":129,"meta":130,"style":130},"python main.py\n",[132,1930,1931],{"__ignoreMap":130},[135,1932,1933,1935],{"class":137,"line":138},[135,1934,192],{"class":141},[135,1936,1937],{"class":145}," main.py\n",[16,1939,1940],{},"You can send a test query using curl to see the pipeline in action.",[125,1942,1944],{"className":127,"code":1943,"language":129,"meta":130,"style":130},"curl --data '{ \"messages\": \"What is the value of X?\"}' http:\u002F\u002Flocalhost:8011\n",[132,1945,1946],{"__ignoreMap":130},[135,1947,1948,1951,1954,1957,1960,1963],{"class":137,"line":138},[135,1949,1950],{"class":141},"curl",[135,1952,1953],{"class":145}," --data",[135,1955,1956],{"class":222}," '",[135,1958,1959],{"class":145},"{ \"messages\": \"What is the value of X?\"}",[135,1961,1962],{"class":222},"'",[135,1964,1965],{"class":145}," http:\u002F\u002Flocalhost:8011\n",[16,1967,1968],{},"You can use more advanced tools like Streamlit to have a UI.",[40,1970,1972],{"id":1971},"what-about-real-time","What about real-time?",[16,1974,1975],{},"Some one of might remember that you were promised \"real-time RAG\", not a regular static one.\nNo worries! The Pathway Live Data Framework automatically adapts to changes, ensuring your RAG stays updated in real time.",[16,1977,1978],{},"With Pathway Live Data Framework, static and real-time data are treated the same way: your pipeline is already working on data stream!",[16,1980,1981],{},"Once your documents are preprocessed and indexed, Pathway Live Data Framework automatically detects any changes in the document directory and updates the document store accordingly.\nThis real-time reactivity ensures that the RAG's responses are always based on the most recent and relevant information available.",[40,1983,1985],{"id":1984},"conclusions-and-pathway-live-data-framework-wrappers","Conclusions and Pathway Live Data Framework wrappers",[16,1987,1988],{},"In this tutorial, you have learned to create a RAG pipeline from scratch with Pathway Live Data Framework: you have implemented the pipeline step-by-step.\nThe RAG is live by default, updating the index whenever the documentation changes.",[16,1990,1991,1992,1996,1997,2001,2002,2006,2007,30],{},"In addition to our ",[26,1993,1995],{"href":1994},"\u002Fdevelopers\u002Ftemplates?tab=ai-pipelines","ready-to-run templates",", Pathway Live Data Framework provides pre-build wrapper to use ",[26,1998,2000],{"href":1999},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fprompts","pre-selected prompts",", ",[26,2003,2005],{"href":2004},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fservers","servers",", or even entire ",[26,2008,2010],{"href":2009},"\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fquestion_answering","RAG pipeline",[16,2012,2013],{},"Instead of doing everything from scratch, take a look at the documentation, you might find what you are trying to do!",[2015,2016,2017],"style",{},"html pre.shiki code .s5Dmg, html code.shiki .s5Dmg{--shiki-default:#FFCB6B}html pre.shiki code .sfyAc, html code.shiki .sfyAc{--shiki-default:#C3E88D}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html pre.shiki code .s6cf3, html code.shiki .s6cf3{--shiki-default:#89DDFF;--shiki-default-font-style:italic}html pre.shiki code .s0W1g, html code.shiki .s0W1g{--shiki-default:#BABED8}html pre.shiki code .sAklC, html code.shiki .sAklC{--shiki-default:#89DDFF}html pre.shiki code .sdLwU, html code.shiki .sdLwU{--shiki-default:#82AAFF}html pre.shiki code .s-wAU, html code.shiki .s-wAU{--shiki-default:#F07178}html pre.shiki code .s7ZW3, html code.shiki .s7ZW3{--shiki-default:#BABED8;--shiki-default-font-style:italic}html pre.shiki code .sx098, html code.shiki .sx098{--shiki-default:#F78C6C}html pre.shiki code .sJ14y, html code.shiki .sJ14y{--shiki-default:#C792EA}html pre.shiki code .saEQR, html code.shiki .saEQR{--shiki-default:#676E95;--shiki-default-font-style:italic}",{"title":130,"searchDepth":213,"depth":213,"links":2019},[2020,2021,2024,2025,2026,2027,2032,2033,2034,2035],{"id":42,"depth":213,"text":43},{"id":119,"depth":213,"text":120,"children":2022},[2023],{"id":178,"depth":243,"text":179},{"id":418,"depth":213,"text":59},{"id":864,"depth":213,"text":865},{"id":1207,"depth":213,"text":1208},{"id":1313,"depth":213,"text":103,"children":2028},[2029,2030,2031],{"id":1319,"depth":243,"text":1320},{"id":1638,"depth":243,"text":1639},{"id":1733,"depth":243,"text":1734},{"id":1872,"depth":213,"text":1873},{"id":1896,"depth":213,"text":1897},{"id":1971,"depth":213,"text":1972},{"id":1984,"depth":213,"text":1985},"Create your own RAG Pipeline with Pathway Live Data Framework","md",{"date":2039,"thumbnail":130,"tags":2040,"keywords":2044},"2025-03-28",[2041,2042,2043],"tutorial","LLM","RAG",[2042,2045,2046,230],"GPT","OpenAI","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fllm-app",{"title":5,"description":2036},{"loc":2047},"2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F20.llm-app","to9SDb5gSgDOJPltn_JztH5Au6XUFka9BIaZn_zyGus",[2053],{"title":2054,"path":2055,"stem":2056,"children":2057,"page":2075},"Developers","\u002Fdevelopers","2.developers",[2058,2922,3244],{"title":2059,"path":2060,"stem":2061,"children":2062,"page":2075},"User Guide","\u002Fdevelopers\u002Fuser-guide","2.developers\u002F4.user-guide",[2063,2117,2386,2503,2581,2694,2826,2842,2901],{"title":2064,"meta":2065,"icon":2066,"path":2067,"stem":2068,"children":2069,"page":2075},"Introduction",{"icon":2066},"heroicons:book-open","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction","2.developers\u002F4.user-guide\u002F10.introduction",[2070,2076,2080,2085,2090,2095,2100,2105,2110],{"title":2071,"path":2072,"stem":2073,"meta":2074},"Welcome","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Fwelcome","2.developers\u002F4.user-guide\u002F10.introduction\u002F10.welcome",{"heading":2075,"toc":2075},false,{"title":2077,"path":158,"stem":2078,"meta":2079},"Installation","2.developers\u002F4.user-guide\u002F10.introduction\u002F20.installation",{"heading":2075,"toc":2075},{"title":2081,"path":2082,"stem":2083,"meta":2084},"Overview","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Flive-data-framework-overview","2.developers\u002F4.user-guide\u002F10.introduction\u002F30.live-data-framework-overview",{},{"title":2086,"path":2087,"stem":2088,"meta":2089},"Starting Examples","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Ffirst-realtime-app","2.developers\u002F4.user-guide\u002F10.introduction\u002F40.first-realtime-app",{},{"title":2091,"path":2092,"stem":2093,"meta":2094},"Core Concepts","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Fconcepts","2.developers\u002F4.user-guide\u002F10.introduction\u002F50.concepts",{},{"title":2096,"path":2097,"stem":2098,"meta":2099},"Why Pathway","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Fwhy-live-data-framework","2.developers\u002F4.user-guide\u002F10.introduction\u002F60.why-live-data-framework",{},{"title":2101,"path":2102,"stem":2103,"meta":2104},"Licensing Guide","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Flicensing-guide","2.developers\u002F4.user-guide\u002F10.introduction\u002F65.licensing-guide",{"heading":2075,"toc":2075},{"title":2106,"path":2107,"stem":2108,"meta":2109},"Streaming and Static Modes","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Fstreaming-and-static-modes","2.developers\u002F4.user-guide\u002F10.introduction\u002F70.streaming-and-static-modes",{},{"title":2111,"path":2112,"stem":2113,"meta":2114},"Batch Processing in Python","\u002Fdevelopers\u002Fuser-guide\u002Fintroduction\u002Fbatch-processing","2.developers\u002F4.user-guide\u002F10.introduction\u002F80.batch-processing",{"thumbnail":2115,"thumbnailFit":2116},"\u002Fassets\u002Fcontent\u002Fdocumentation\u002Fpathway-batch.png","contain",{"title":2118,"path":2119,"stem":2120,"children":2121,"page":2075},"Connect","\u002Fdevelopers\u002Fuser-guide\u002Fconnect","2.developers\u002F4.user-guide\u002F20.connect",[2122,2134,2138,2143,2147,2152,2165,2175,2180,2201],{"title":2123,"path":2124,"stem":2125,"meta":2126},"Basic Data Types","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fdatatypes","2.developers\u002F4.user-guide\u002F20.connect\u002F20.datatypes",{"date":2127,"tags":2128,"keywords":2130,"notebook_export_path":2133},"2024-01-26",[2041,2129],"table",[2131,2132],"type","schema","notebooks\u002Ftutorials\u002Fbasic_datatypes.ipynb",{"title":2135,"path":424,"stem":2136,"meta":2137},"Connectors in Pathway","2.developers\u002F4.user-guide\u002F20.connect\u002F30.connectors-overview",{},{"title":2139,"path":515,"stem":2140,"meta":2141},"Connectors list","2.developers\u002F4.user-guide\u002F20.connect\u002F35.live-data-framework-connectors",{"layout":2142,"aside":405,"toc":2075,"single":405},"default",{"title":2144,"path":935,"stem":2145,"meta":2146},"Defining Schema","2.developers\u002F4.user-guide\u002F20.connect\u002F40.schema",{},{"title":2148,"path":2149,"stem":2150,"meta":2151},"Generate Data Streams with the demo Module","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fartificial-streams","2.developers\u002F4.user-guide\u002F20.connect\u002F50.artificial-streams",{},{"title":2153,"path":2154,"stem":2155,"meta":2156},"Automatic Generation of Schema Class","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fschema-generation","2.developers\u002F4.user-guide\u002F20.connect\u002F60.schema-generation",{"date":2157,"thumbnail":2158,"tags":2160,"keywords":2161,"notebook_export_path":2164},"2023-11-08",{"src":2159},"\u002Fassets\u002Fcontent\u002Fblog\u002Fth-custom-connectors.png",[2041,2129],[2132,2131,2162,2163],"JSON","connectors","notebooks\u002Ftutorials\u002Fschema-generation.ipynb",{"title":2166,"path":2167,"stem":2168,"meta":2169},"Dealing with JSON Data Type","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fjson_type","2.developers\u002F4.user-guide\u002F20.connect\u002F70.json_type",{"date":2170,"thumbnail":2171,"tags":2172,"keywords":2173,"notebook_export_path":2174},"2023-12-22","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-json.png",[2041,2129],[2162,2131,2132],"notebooks\u002Ftutorials\u002Fjson_type.ipynb",{"title":2176,"path":2177,"stem":2178,"meta":2179},"Switching from Batch to Streaming","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fswitch-from-batch-to-streaming","2.developers\u002F4.user-guide\u002F20.connect\u002F80.switch-from-batch-to-streaming",{},{"title":2181,"path":2182,"stem":2183,"meta":2184},"Live Web Scraping","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fpython-web-scraping","2.developers\u002F4.user-guide\u002F20.connect\u002F90.python-web-scraping",{"date":2185,"tags":2186,"keywords":2187,"tech_icons":2195,"thumbnail":2197,"deployment_tag":2198,"docker_github_link":2200},"2025-03-13",[2041,2163],[2188,2189,2190,2191,2192,2193,2194],"connector","scraping","WEB","Python","input","streaming","web scraping",[2196],"streamline:web","\u002Fassets\u002Fcontent\u002Fshowcases\u002Fweb_scraping\u002Fweb-scraping-th.png",[2199],"docker","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fweb-scraping",{"title":2202,"path":2203,"stem":2204,"children":2205,"page":2075},"Connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors",[2206,2222,2234,2246,2255,2274,2285,2298,2312,2320,2332,2342,2351,2363,2376],{"title":2207,"path":442,"stem":2208,"meta":2209},"File System Connector","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F05.fs-connector",{"date":2127,"tags":2210,"tech_icons":2211,"deployment_tag":2213,"keywords":2215,"notebook_export_path":2221},[2041,2129],[2212],"lets-icons:file-fill",[2214,2199],"jupyter",[2188,2216,2217,2218,2192,2219,2220,2193],"file system","csv","json","output","static","notebooks\u002Ftutorials\u002Ffs_connector.ipynb",{"title":2223,"path":2224,"stem":2225,"meta":2226},"CSV connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fcsv_connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F10.csv_connectors",{"date":2227,"thumbnail":2228,"tags":2229,"keywords":2230,"tech_icons":2232},"2023-01-12","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-csv-connectors.png",[2041,2163],[2188,2231,2192,2219,2220,2193],"CSV",[2233],"mdi:file-csv",{"title":2235,"path":2236,"stem":2237,"meta":2238},"Sending alerts to Slack","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fslack_send_alerts","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F100.slack_send_alerts",{"date":2239,"tags":2240,"keywords":2241,"tech_icons":2244},"2024-02-16",[2041,2163],[2188,2242,2243,2219,2193],"Slack","alerts",[2245],"mdi:slack",{"title":2247,"path":2248,"stem":2249,"meta":2250},"Airbyte connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fairbyte-connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F110.airbyte-connectors",{"tags":2251,"keywords":2252},[2041,2163],[2188,2253,2254,2192,2220,2193],"Airbyte","cloud",{"title":2256,"path":2257,"stem":2258,"meta":2259},"Using Pathway Debezium Connector for MongoDB","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fmongodb-debezium","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F115.mongodb-debezium",{"date":2260,"tags":2261,"keywords":2262,"tech_icons":2270,"deployment_tag":2272,"docker_github_link":2273,"hideInConnectorList":405},"2024-10-30",[2041,2163],[2188,2263,2264,2265,2266,2267,2268,2269,2192,2219,2193],"database","MongoDB","CDC","Debezium","Kafka","Zookeeper","Docker",[2271],"material-symbols-light:database",[2199],"https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fdebezium-mongodb-example",{"title":2275,"path":2276,"stem":2277,"meta":2278},"JSON Lines connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fjsonlines-connector","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F15.jsonlines-connector",{"date":2279,"tags":2280,"keywords":2281,"tech_icons":2283},"2024-02-20",[2041,2163],[2188,2282,2162,2192,2219,2220,2193],"JSON Lines",[2284],"lets-icons:json",{"title":2286,"path":2287,"stem":2288,"meta":2289},"Database connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fdatabase-connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F20.database-connectors",{"date":2290,"thumbnail":2291,"tags":2292,"keywords":2293,"tech_icons":2295,"deployment_tag":2296,"docker_github_link":2297},"2023-01-19","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-database-connectors.png",[2041,2163],[2188,2263,2294,2265,2266,2267,2268,2269,2192,2219,2193],"PostgreSQL",[2271],[2199],"https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fdebezium-postgres-example",{"title":2299,"path":2300,"stem":2301,"meta":2302},"Connecting Pathway to NeonDB","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fneondb-connector","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F23.neondb-connector",{"date":2303,"tags":2304,"keywords":2305,"tech_icons":2311,"hideInConnectorList":405},"2026-06-03",[2041,2163],[2188,2263,2306,2307,2294,2308,2265,2309,2310,2192,2219,2193],"NeonDB","Neon","serverless","logical replication","SSL",[2271],{"title":2313,"path":2314,"stem":2315,"meta":2316},"Custom Python connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fcustom-python-connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F30.custom-python-connectors",{"date":2317,"thumbnail":2159,"tags":2318,"keywords":2319},"2023-04-13",[2041,2163],[2188,2191,2192,2193],{"title":2321,"path":2322,"stem":2323,"meta":2324},"Kafka connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fkafka_connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F30.kafka_connectors",{"date":2325,"thumbnail":2326,"tags":2327,"tech_icons":2328,"deployment_tag":2330,"keywords":2331},"2023-01-15","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-kafka-connectors.png",[2041,2163],[2329],"mdi:apache-kafka",[2199],[2188,2267,2269,2192,2219,2193],{"title":2333,"path":2334,"stem":2335,"meta":2336},"NATS connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fnats-connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F33.nats-connectors",{"date":2337,"tags":2338,"deployment_tag":2339,"keywords":2340},"2024-12-06",[2041,2163],[2199],[2188,2341,2269,2192,2219,2193],"NATS",{"title":2343,"path":2344,"stem":2345,"meta":2346},"Python output connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fpython-output-connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F35.python-output-connectors",{"date":2347,"thumbnail":2348,"tags":2349,"keywords":2350},"2023-08-31","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-output-connector.png",[2041,2163],[2188,2191,2219,2193],{"title":2352,"path":2353,"stem":2354,"meta":2355},"Google Drive connector","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fgdrive-connector","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F70.gdrive-connector",{"date":2356,"thumbnail":2357,"tags":2358,"keywords":2359,"tech_icons":2361},"2023-12-14","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-gdrive-connector.png",[2041,2163],[2188,2360,2254,2192,2220,2193],"Google Drive",[2362],"simple-icons:googledrive",{"title":2364,"path":2365,"stem":2366,"meta":2367},"Switching from Kafka to Redpanda","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fswitching-to-redpanda","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F80.switching-to-redpanda",{"date":2368,"thumbnail":2369,"tags":2370,"keywords":2371,"deployment_tag":2373,"tech_icons":2374,"docker_github_link":2375,"hideInConnectorList":405},"2023-03-24","\u002Fassets\u002Fcontent\u002Ftutorials\u002Fredpanda\u002Fth-Kafka_vs_Redpanda.png",[2041,2163],[2188,2372,2267,2268,2269,2192,2219,2193],"Redpanda",[2199],[2329],"https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fbest-movies-example\u002Fredpanda-version",{"title":2377,"path":2378,"stem":2379,"meta":2380},"Custom WebSockets connectors","\u002Fdevelopers\u002Fuser-guide\u002Fconnect\u002Fconnectors\u002Fwebsockets-connectors","2.developers\u002F4.user-guide\u002F20.connect\u002F99.connectors\u002F90.websockets-connectors",{"date":2381,"tags":2382,"keywords":2383,"tech_icons":2385},"2023-01-24",[2041,2163],[2188,2384,2191,2192,2219,2193],"WebSockets",[2196],{"title":2387,"meta":2388,"path":2389,"stem":2390,"children":2391,"page":2075},"Transform",{},"\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation","2.developers\u002F4.user-guide\u002F30.data-transformation",[2392,2397,2410,2416,2422,2435,2451,2470,2486,2491],{"title":2393,"path":2394,"stem":2395,"meta":2396},"Table Operations Overview","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Ftable-operations","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F10.table-operations",{},{"title":2398,"path":2399,"stem":2400,"meta":2401},"Asynchronous Transformations","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Fasynchronous-transformations","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F100.asynchronous-transformations",{"date":2279,"thumbnail":130,"tags":2402,"keywords":2404,"notebook_export_path":2409},[2041,2403],"engineering",[192,2405,2406,2407,1081,2408],"function","asynchronous","transformation","AsyncTransformer","notebooks\u002Ftutorials\u002Fasynctransformer.ipynb",{"title":2411,"path":2412,"stem":2413,"meta":2414},"Simple Join","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Fjoin-manual","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F20.join-manual",{"notebook_export_path":2415},"notebooks\u002Ftutorials\u002Fjoin_manual.ipynb",{"title":2417,"path":2418,"stem":2419,"meta":2420},"Groupby Reduce","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Fgroupby-reduce-manual","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F30.groupby-reduce-manual",{"notebook_export_path":2421},"notebooks\u002Ftutorials\u002Fgroupby_reduce_manual.ipynb",{"title":2423,"path":2424,"stem":2425,"meta":2426},"Writing Stateful Reducers","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Fcustom-reducers","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F40.custom-reducers",{"date":2427,"thumbnail":2171,"tags":2428,"keywords":2429,"notebook_export_path":2434},"2024-01-17",[2041],[2430,2431,2432,2433],"reducers","aggregate","sumofsquares","median","notebooks\u002Ftutorials\u002Fcustom_reducers.ipynb",{"title":2436,"path":2437,"stem":2438,"meta":2439},"Iterative Computations","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Fiterate","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F50.iterate",{"date":2440,"thumbnail":130,"tags":2441,"keywords":2442,"notebook_export_path":2450},"2023-12-28",[2041,2403],[2443,2444,2445,2446,2447,2448,2449],"iterate","iteration","iterative","updates","connected components","collatz conjecture","newton method","notebooks\u002Ftutorials\u002Fiterate.ipynb",{"title":2452,"path":2453,"stem":2454,"children":2455,"meta":2467},"Indexes in Pathway Live Data Framework","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Findexes","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F60.indexes",[2456],{"title":2452,"path":2453,"stem":2454,"meta":2457},{"date":2458,"thumbnail":2459,"tags":2460,"keywords":2461,"notebook_export_path":2466},"2023-11-15","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-computing-pagerank.png",[2041,2403],[2462,230,1424,2463,2464,2465],"index","asof join","asof_now","KNN","notebooks\u002Ftutorials\u002Findexes.ipynb",{"date":2458,"thumbnail":2459,"tags":2468,"keywords":2469,"notebook_export_path":2466},[2041,2403],[2462,230,1424,2463,2464,2465],{"title":2471,"path":2472,"stem":2473,"children":2474,"meta":2483},"Indexing from 'groupby' and Single-row Tables","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Findexing-grouped-tables","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F70.indexing-grouped-tables",[2475],{"title":2471,"path":2472,"stem":2473,"meta":2476},{"date":2477,"thumbnail":2459,"tags":2478,"keywords":2479},"2023-03-31",[2041,2129],[2480,2481,2482],"groupby","ix_ref","single-row table",{"date":2477,"thumbnail":2459,"tags":2484,"keywords":2485},[2041,2129],[2480,2481,2482],{"title":2487,"path":2488,"stem":2489,"meta":2490},"SQL API","\u002Fdevelopers\u002Fuser-guide\u002Fdata-transformation\u002Fsql","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F80.sql",{"redirection":405},{"title":2492,"path":1448,"stem":2493,"meta":2494},"User-defined Functions","2.developers\u002F4.user-guide\u002F30.data-transformation\u002F90.user-defined-functions",{"date":2495,"thumbnail":130,"tags":2496,"keywords":2497,"notebook_export_path":2502},"2024-02-21",[2041,2403],[192,2498,2405,2499,2407,2500,2501],"udf","apply","cache","timeout","notebooks\u002Ftutorials\u002Fudf.ipynb",{"title":2504,"path":2505,"stem":2506,"children":2507,"page":2075},"Temporal Data","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data","2.developers\u002F4.user-guide\u002F40.temporal-data",[2508,2514,2519,2534,2546,2558,2572],{"title":2509,"path":2510,"stem":2511,"meta":2512},"Windowby Reduce","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data\u002Fwindows-manual","2.developers\u002F4.user-guide\u002F40.temporal-data\u002F10.windows-manual",{"notebook_export_path":2513},"notebooks\u002Ftutorials\u002Fwindowby_manual.ipynb",{"title":2515,"path":2516,"stem":2517,"meta":2518},"Late Data and Cutoffs","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data\u002Fbehaviors","2.developers\u002F4.user-guide\u002F40.temporal-data\u002F20.behaviors",{},{"title":2520,"path":2521,"stem":2522,"meta":2523},"Controlling Temporal Behavior of Windows","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data\u002Fwindows_with_behaviors","2.developers\u002F4.user-guide\u002F40.temporal-data\u002F30.windows_with_behaviors",{"date":2524,"thumbnail":2525,"keywords":2526,"notebook_export_path":2533},"2024-01-08","assets\u002Fcontent\u002Ftutorials\u002Fclickstream_window_join\u002Fclickstream-window-join-th.png",[2527,2528,2529,2530,2531,2532],"windowby","behavior","late data","delay","cutoff","out-of-order data","notebooks\u002Ftutorials\u002Fwindows_temporal_behavior.ipynb",{"title":2535,"path":2536,"stem":2537,"meta":2538},"ASOF Join","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data\u002Fasof-join","2.developers\u002F4.user-guide\u002F40.temporal-data\u002F40.asof-join",{"date":2539,"thumbnail":2540,"tags":2541,"keywords":2542,"notebook_export_path":2545},"2023-05-24","\u002Fassets\u002Fcontent\u002Ftutorials\u002Ffinance_ts_asof_join\u002Fasof-join-tutorial-th.png",[2041,2403],[2463,2543,2544],"stock prices","financial data analysis","notebooks\u002Ftutorials\u002Fasof_join.ipynb",{"title":2547,"path":2548,"stem":2549,"meta":2550},"Interval Join","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data\u002Finterval-join","2.developers\u002F4.user-guide\u002F40.temporal-data\u002F50.interval-join",{"date":2551,"thumbnail":2552,"tags":2553,"keywords":2554,"notebook_export_path":2557},"2023-05-22","assets\u002Fcontent\u002Ftutorials\u002Ffleet_eta_interval_join\u002Ffleet-eta-interval-join-th.png",[2041,2403],[2555,2556],"interval join","ETA","notebooks\u002Ftutorials\u002Finterval_join.ipynb",{"title":2559,"path":2560,"stem":2561,"meta":2562},"Window Join","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data\u002Fwindow-join","2.developers\u002F4.user-guide\u002F40.temporal-data\u002F60.window-join",{"date":2563,"thumbnail":2564,"tags":2565,"keywords":2566,"notebook_export_path":2571},"2023-06-14","\u002Fassets\u002Fcontent\u002Ftutorials\u002Fclickstream_window_join\u002Fclickstream-window-join-th.png",[2041,2403],[2567,2568,2569,2570],"window","behavior analysis","clickstream","conversion rate","notebooks\u002Ftutorials\u002Fwindow_join.ipynb",{"title":2573,"path":2574,"stem":2575,"meta":2576},"Controlling Temporal Behavior of Interval Join","\u002Fdevelopers\u002Fuser-guide\u002Ftemporal-data\u002Ftemporal_behavior","2.developers\u002F4.user-guide\u002F40.temporal-data\u002F70.temporal_behavior",{"date":2577,"thumbnail":2552,"tags":2578,"keywords":2579,"notebook_export_path":2580},"2023-11-28",[2041,2403],[2555,2528,2529,2530,2531,2532],"notebooks\u002Ftutorials\u002Finterval_join_temporal_behavior.ipynb",{"title":2582,"meta":2583,"path":2584,"stem":2585,"children":2586,"page":2075},"LLM tooling",{},"\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack","2.developers\u002F4.user-guide\u002F50.llm-xpack",[2587,2596,2600,2609,2622,2634,2646,2658,2668,2676,2685],{"title":2588,"path":2589,"stem":2590,"meta":2591},"Introduction to the LLM xpack","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Foverview","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F10.overview",{"date":2592,"thumbnail":130,"tags":2593,"keywords":2594},"2024-02-14",[2041,2042],[2042,2045,2046,2595],"Gemini",{"title":5,"path":2047,"stem":2050,"meta":2597},{"date":2039,"thumbnail":130,"tags":2598,"keywords":2599},[2041,2042,2043],[2042,2045,2046,230],{"title":2601,"path":2602,"stem":2603,"meta":2604},"Pathway Live Data Framework as a Vector Store","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Flive-data-framework-vector-store","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F25.live-data-framework-vector-store",{"date":2605,"thumbnail":130,"tags":2606,"keywords":2607},"2025-05-07",[2041,2042,2043],[2042,2045,2046,230,2608],"Vector Store",{"title":59,"path":860,"stem":2610,"children":2611,"meta":2619},"2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F30.docs-indexing",[2612],{"title":59,"path":860,"stem":2610,"meta":2613},{"date":2614,"thumbnail":130,"tags":2615,"keywords":2618},"2025-01-13",[2041,230,2616,2617],"document storage","retrieval",[2042,2045,2046,2595],{"date":2614,"thumbnail":130,"tags":2620,"keywords":2621},[2041,230,2616,2617],[2042,2045,2046,2595],{"title":2623,"path":2624,"stem":2625,"meta":2626},"Pathway Live Data Framework MCP Server","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Flive-data-framework-mcp-server","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F40.live-data-framework-mcp-server",{"date":2627,"thumbnail":130,"tags":2628,"keywords":2629},"2025-08-01",[2041,2403],[2630,2631,2632,2633],"MCP","MCP server","API","protocol",{"title":2635,"path":2636,"stem":2637,"meta":2638},"Claude Desktop","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Flive-data-framework-mcp-claude-desktop","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F41.live-data-framework-mcp-claude-desktop",{"date":2639,"thumbnail":130,"tags":2640,"keywords":2642},"2025-08-19",[2041,2403,2641],"claude",[2630,2635,2643,2644,2645],"real-time data","document store","statistics",{"title":2647,"path":2648,"stem":2649,"meta":2650},"Parsers","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fparsers","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F50.parsers",{"date":2651,"tags":2652,"keywords":2653},"2024-06-04",[2041,2042],[2042,2654,2655,2656,2657],"unstructured","docling","parsers","ocr",{"title":2659,"path":2660,"stem":2661,"meta":2662},"Chunking text","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fsplitters","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F60.splitters",{"date":2663,"thumbnail":130,"tags":2664,"keywords":2667},"2025-02-04",[2665,2666],"splitters","chunking",[2656,2666],{"title":2669,"path":2670,"stem":2671,"meta":2672},"Embedders","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Fembedders","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F70.embedders",{"date":2663,"thumbnail":130,"tags":2673,"keywords":2674},[2041,690],[2042,2045,2046,2595,2675,549],"LiteLLM",{"title":2677,"path":1645,"stem":2678,"meta":2679},"LLM Chats","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F80.llm-chats",{"date":2680,"thumbnail":130,"tags":2681,"keywords":2683},"2025-01-30",[2041,2042,2682,2677],"LLM Wrappers",[2042,2045,2046,2595,2675,2684],"Wrapper",{"title":2686,"path":2687,"stem":2688,"meta":2689},"Rerankers","\u002Fdevelopers\u002Fuser-guide\u002Fllm-xpack\u002Frerankers","2.developers\u002F4.user-guide\u002F50.llm-xpack\u002F90.rerankers",{"date":2663,"thumbnail":130,"tags":2690,"keywords":2692},[2041,2691],"reranker",[2042,2693],"Reranker",{"title":2695,"meta":2696,"path":2697,"stem":2698,"children":2699,"page":2075},"Deploy",{},"\u002Fdevelopers\u002Fuser-guide\u002Fdeployment","2.developers\u002F4.user-guide\u002F60.deployment",[2700,2705,2710,2727,2743,2748,2753,2758,2765,2770,2775,2788,2800,2805,2816],{"title":2701,"path":2702,"stem":2703,"meta":2704},"Cloud Deployment of Pathway Live Data Framework","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fcloud-deployment","2.developers\u002F4.user-guide\u002F60.deployment\u002F10.cloud-deployment",{},{"title":2706,"path":2707,"stem":2708,"meta":2709},"Deploy to GCP","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fgcp-deploy","2.developers\u002F4.user-guide\u002F60.deployment\u002F15.gcp-deploy",{},{"title":2711,"path":2712,"stem":2713,"meta":2714},"Deploy to AWS Cloud","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Faws-fargate-deploy","2.developers\u002F4.user-guide\u002F60.deployment\u002F20.aws-fargate-deploy",{"date":2715,"tags":2716,"thumbnail":2719,"keywords":2721,"docker_github_link":2726,"deployButtons":2075},"2024-08-16",[2717,2718],"showcase","data-pipeline",{"src":2720},"\u002Fassets\u002Fcontent\u002Fdocumentation\u002Faws\u002Faws-fargate-overview-th.png",[2722,2723,2724,2269,2725],"AWS","Fargate","cloud deployment","AWS Fargate","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Faws-fargate-deploy",{"title":2728,"path":2729,"stem":2730,"meta":2731},"Deploy to Azure","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fazure-aci-deploy","2.developers\u002F4.user-guide\u002F60.deployment\u002F25.azure-aci-deploy",{"date":2732,"tags":2733,"thumbnail":2734,"keywords":2736,"docker_github_link":2742,"deployButtons":2075},"2024-11-20",[2717,2718],{"src":2735,"contain":405},"\u002Fassets\u002Fcontent\u002Fdocumentation\u002Fazure\u002Fazure-aci-overview-th.png",[2737,2738,2724,2269,2739,2740,2741],"Azure","ACI","Azure ACI","Azure Marketplace","deployment","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fazure-aci-deploy",{"title":2744,"path":2745,"stem":2746,"meta":2747},"Deploy with Render","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Frender-deploy","2.developers\u002F4.user-guide\u002F60.deployment\u002F30.render-deploy",{},{"title":2749,"path":2750,"stem":2751,"meta":2752},"Deploy to Nebius AI Cloud","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fnebius-deploy","2.developers\u002F4.user-guide\u002F60.deployment\u002F32.nebius-deploy",{},{"title":2754,"path":2755,"stem":2756,"meta":2757},"Notebook-to-Docker Conversion","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fjupyter-docker","2.developers\u002F4.user-guide\u002F60.deployment\u002F35.jupyter-docker",{},{"title":2759,"path":2760,"stem":2761,"meta":2762},"Tutorial: From Jupyter to Deploy","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Ffrom-jupyter-to-deploy","2.developers\u002F4.user-guide\u002F60.deployment\u002F40.from-jupyter-to-deploy",{"date":2763,"project_github_link":2764},"2023-11-29","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Ffrom_jupyter_to_deploy",{"title":2766,"path":2767,"stem":2768,"meta":2769},"CI\u002FCD","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fintegration","2.developers\u002F4.user-guide\u002F60.deployment\u002F45.integration",{},{"title":2771,"path":2772,"stem":2773,"meta":2774},"Docker Deployment of Pathway Live Data Framework","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fdocker-deployment","2.developers\u002F4.user-guide\u002F60.deployment\u002F5.docker-deployment",{},{"title":2776,"path":2777,"stem":2778,"meta":2779},"Monitoring a Pathway Live Data Framework Instance","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Flive-data-framework-monitoring","2.developers\u002F4.user-guide\u002F60.deployment\u002F50.live-data-framework-monitoring",{"date":2780,"thumbnail":2781,"tags":2782,"keywords":2783},"2024-05-27","\u002Fassets\u002Fcontent\u002Fblog\u002Fth-prometheus.png",[2041,2403],[2784,2785,2786,2787],"monitoring","OpenTelemetry","Grafana","metrics",{"title":2789,"path":2790,"stem":2791,"meta":2792},"Pathway Live Data Framework Web Dashboard","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fweb-dashboard","2.developers\u002F4.user-guide\u002F60.deployment\u002F52.web-dashboard",{"date":2793,"tags":2794,"keywords":2795},"2026-01-13",[2041,2403],[2796,2797,2784,2798,2799,2787],"Web Dashboard","UI","graph","charts",{"title":2801,"path":2802,"stem":2803,"meta":2804},"Persistence in Pathway Live Data Framework","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fpersistence","2.developers\u002F4.user-guide\u002F60.deployment\u002F55.persistence",{},{"title":2806,"path":2807,"stem":2808,"meta":2809},"Persistence in Pathway Live Data Framework: How to Resume the Interrupted Computation","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fpersistence_recovery","2.developers\u002F4.user-guide\u002F60.deployment\u002F60.persistence_recovery",{"date":2810,"thumbnail":130,"tags":2811,"keywords":2813},"2023-11-02",[2041,2403,2812],"dataflow management",[2814,2815],"persistence","recovery",{"title":2817,"path":2818,"stem":2819,"meta":2820},"Persistence in Pathway Live Data Framework: How to Add New Data without Full Computation Restart","\u002Fdevelopers\u002Fuser-guide\u002Fdeployment\u002Fpersistence_restart_with_new_data","2.developers\u002F4.user-guide\u002F60.deployment\u002F65.persistence_restart_with_new_data",{"date":2821,"thumbnail":130,"tags":2822,"keywords":2823},"2023-11-14",[2041,2403,2812],[2814,2824,2825],"optimization","restart",{"title":2827,"path":2828,"stem":2829,"children":2830,"page":2075},"Migrating","\u002Fdevelopers\u002Fuser-guide\u002Fmigrating","2.developers\u002F4.user-guide\u002F70.migrating",[2831,2837],{"title":2832,"path":2833,"stem":2834,"meta":2835},"Coming from Pandas","\u002Fdevelopers\u002Fuser-guide\u002Fmigrating\u002Fmigrate-from-pandas","2.developers\u002F4.user-guide\u002F70.migrating\u002F10.migrate-from-pandas",{"notebook_export_path":2836},"notebooks\u002Ftutorials\u002Ffrom_pandas.ipynb",{"title":2838,"path":2839,"stem":2840,"meta":2841},"Pandas Cheat Sheet","\u002Fdevelopers\u002Fuser-guide\u002Fmigrating\u002Fpandas-cheat-sheet","2.developers\u002F4.user-guide\u002F70.migrating\u002F20.pandas-cheat-sheet",{},{"title":2843,"path":2844,"stem":2845,"children":2846,"page":2075},"Advanced","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced","2.developers\u002F4.user-guide\u002F80.advanced",[2847,2852,2862,2868,2881,2886,2891,2896],{"title":2848,"path":2849,"stem":2850,"meta":2851},"Architecture","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Fworker-architecture","2.developers\u002F4.user-guide\u002F80.advanced\u002F10.worker-architecture",{},{"title":2853,"path":2854,"stem":2855,"meta":2856},"Setting up Cache in Asynchronous Transformations","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Ffunction_calls_caching","2.developers\u002F4.user-guide\u002F80.advanced\u002F20.function_calls_caching",{"date":2857,"thumbnail":130,"tags":2858,"keywords":2859},"2023-11-30",[2041,2403,2812],[2814,2824,2860,2861],"async_transformer","caching",{"title":2863,"path":2864,"stem":2865,"meta":2866},"Data Model Consistency","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Fconsistency","2.developers\u002F4.user-guide\u002F80.advanced\u002F30.consistency",{"notebook_export_path":2867},"notebooks\u002Ftutorials\u002Fconsistency.ipynb",{"title":2869,"path":2870,"stem":2871,"meta":2872},"Writing declarative over imperative pipelines","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Fdeclarative_vs_imperative","2.developers\u002F4.user-guide\u002F80.advanced\u002F40.declarative_vs_imperative",{"date":2873,"thumbnail":130,"tags":2874,"keywords":2875,"notebook_export_path":2880},"2025-11-25",[2041,2403],[2876,2877,2878,2879,2443],"declarative","imperative","ordered","splitting","notebooks\u002Ftutorials\u002Fdeclarative_vs_imperative.ipynb",{"title":2882,"path":2883,"stem":2884,"meta":2885},"Best Practices","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Fbest-practices","2.developers\u002F4.user-guide\u002F80.advanced\u002F5.best-practices",{},{"title":2887,"path":2888,"stem":2889,"meta":2890},"How Pathway Live Data Framework Connectors Work","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Fhow-live-data-framework-connectors-work","2.developers\u002F4.user-guide\u002F80.advanced\u002F50.how-live-data-framework-connectors-work",{},{"title":2892,"path":2893,"stem":2894,"meta":2895},"Dynamic Worker Scaling","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Fworker_count_scaling","2.developers\u002F4.user-guide\u002F80.advanced\u002F60.worker_count_scaling",{},{"title":2897,"path":2898,"stem":2899,"meta":2900},"Running on Multiple Machines","\u002Fdevelopers\u002Fuser-guide\u002Fadvanced\u002Frunning_on_multiple_machines","2.developers\u002F4.user-guide\u002F80.advanced\u002F70.running_on_multiple_machines",{},{"title":2902,"meta":2903,"path":2904,"stem":2905,"children":2906,"page":2075},"Help And Updates",{},"\u002Fdevelopers\u002Fuser-guide\u002Fdevelopment","2.developers\u002F4.user-guide\u002F90.development",[2907,2912,2917],{"title":2908,"path":2909,"stem":2910,"meta":2911},"Troubleshooting","\u002Fdevelopers\u002Fuser-guide\u002Fdevelopment\u002Ftroubleshooting","2.developers\u002F4.user-guide\u002F90.development\u002F10.troubleshooting",{},{"title":2913,"path":2914,"stem":2915,"meta":2916},"Changelog","\u002Fdevelopers\u002Fuser-guide\u002Fdevelopment\u002Fchangelog","2.developers\u002F4.user-guide\u002F90.development\u002F20.changelog",{"toc":2075},{"title":2918,"path":2919,"stem":2920,"meta":2921},"Get help","\u002Fdevelopers\u002Fuser-guide\u002Fdevelopment\u002Fget-help","2.developers\u002F4.user-guide\u002F90.development\u002F30.get-help",{},{"title":2923,"meta":2924,"path":2925,"stem":2926,"children":2927,"page":2075},"API docs",{},"\u002Fdevelopers\u002Fapi-docs","2.developers\u002F5.API-docs",[2928,2933,2938,2944,2949,2954,2959,3023,3028,3033,3038,3046,3234,3239],{"title":2929,"path":2930,"stem":2931,"meta":2932},"Pathway Live Data Framework API","\u002Fdevelopers\u002Fapi-docs\u002Fpathway","2.developers\u002F5.API-docs\u002F1.pathway",{"sidebar":2632},{"title":2934,"path":2935,"stem":2936,"meta":2937},"pw.reducers","\u002Fdevelopers\u002Fapi-docs\u002Freducers","2.developers\u002F5.API-docs\u002F10.reducers",{"sidebar":2632},{"title":2939,"path":2940,"stem":2941,"meta":2942},"pw.sql","\u002Fdevelopers\u002Fapi-docs\u002Fsql-api","2.developers\u002F5.API-docs\u002F11.SQL-API",{"notebook_export_path":2943},"notebooks\u002Ftutorials\u002Fsql_api.ipynb",{"title":2945,"path":2946,"stem":2947,"meta":2948},"pw.temporal","\u002Fdevelopers\u002Fapi-docs\u002Ftemporal","2.developers\u002F5.API-docs\u002F12.temporal",{"sidebar":2632},{"title":2950,"path":2951,"stem":2952,"meta":2953},"pw.udfs","\u002Fdevelopers\u002Fapi-docs\u002Fudfs","2.developers\u002F5.API-docs\u002F13.udfs",{"sidebar":2632},{"title":2955,"path":2956,"stem":2957,"meta":2958},"pw.xpacks.connectors","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-sharepoint","2.developers\u002F5.API-docs\u002F14.pathway-xpacks-sharepoint",{"sidebar":2632},{"title":2960,"path":2961,"stem":2962,"children":2963,"meta":3022},"pw.xpacks.llm","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002Findex",[2964,2966,2971,2975,2980,2985,2990,2995,3000,3005,3009,3013,3017],{"title":2960,"path":2961,"stem":2962,"meta":2965},{"sidebar":2632},{"title":2967,"path":2968,"stem":2969,"meta":2970},"pw.xpacks.llm.llms","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fllms","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F1.llms",{"sidebar":2632},{"title":2972,"path":2004,"stem":2973,"meta":2974},"pw.xpacks.llm.servers","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F10.servers",{"sidebar":2632},{"title":2976,"path":2977,"stem":2978,"meta":2979},"pw.xpacks.llm.utils","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Futils","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F11.utils",{"sidebar":2632},{"title":2981,"path":2982,"stem":2983,"meta":2984},"pw.xpacks.llm.mcp_server","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fmcp_server","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F12.mcp_server",{"sidebar":2632},{"title":2986,"path":2987,"stem":2988,"meta":2989},"pw.xpacks.llm.embedders","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fembedders","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F2.embedders",{"sidebar":2632},{"title":2991,"path":2992,"stem":2993,"meta":2994},"pw.xpacks.llm.parsers","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fparsers","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F3.parsers",{"sidebar":2632},{"title":2996,"path":2997,"stem":2998,"meta":2999},"pw.xpacks.llm.splitters","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fsplitters","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F4.splitters",{"sidebar":2632},{"title":3001,"path":3002,"stem":3003,"meta":3004},"pw.xpacks.llm.vector_store","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Fvectorstore","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F5.vectorstore",{"sidebar":2632},{"title":3006,"path":522,"stem":3007,"meta":3008},"pw.xpacks.llm.document_store","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F6.document_store",{"sidebar":2632},{"title":3010,"path":2009,"stem":3011,"meta":3012},"pw.xpacks.llm.question_answering","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F7.question_answering",{"sidebar":2632},{"title":3014,"path":1999,"stem":3015,"meta":3016},"pw.xpacks.llm.prompts","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F8.prompts",{"sidebar":2632},{"title":3018,"path":3019,"stem":3020,"meta":3021},"pw.xpacks.llm.rerankers","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-xpacks-llm\u002Frerankers","2.developers\u002F5.API-docs\u002F15.pathway-xpacks-llm\u002F9.rerankers",{"sidebar":2632},{"sidebar":2632},{"title":3024,"path":3025,"stem":3026,"meta":3027},"pw.Table","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-table","2.developers\u002F5.API-docs\u002F2.pathway-table",{"sidebar":2632},{"title":3029,"path":3030,"stem":3031,"meta":3032},"pw.debug","\u002Fdevelopers\u002Fapi-docs\u002Fdebug","2.developers\u002F5.API-docs\u002F4.debug",{"sidebar":2632},{"title":3034,"path":3035,"stem":3036,"meta":3037},"pw.demo","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-demo","2.developers\u002F5.API-docs\u002F5.pathway-demo",{"sidebar":2632},{"title":3039,"path":3040,"stem":3041,"children":3042,"meta":3045},"pw.indexing","\u002Fdevelopers\u002Fapi-docs\u002Findexing","2.developers\u002F5.API-docs\u002F6.indexing",[3043],{"title":3039,"path":3040,"stem":3041,"meta":3044},{"sidebar":2632},{"sidebar":2632},{"title":3047,"path":3048,"stem":3049,"children":3050,"meta":3233},"pw.io","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io","2.developers\u002F5.API-docs\u002F7.pathway-io\u002Findex",[3051,3053,3058,3063,3068,3073,3078,3083,3088,3093,3098,3103,3108,3113,3118,3123,3128,3133,3138,3143,3148,3153,3158,3163,3168,3173,3178,3183,3188,3193,3198,3203,3208,3213,3218,3223,3228],{"title":3047,"path":3048,"stem":3049,"meta":3052},{"sidebar":2632},{"title":3054,"path":3055,"stem":3056,"meta":3057},"pw.io.airbyte","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fairbyte","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F110.airbyte",{"sidebar":2632},{"title":3059,"path":3060,"stem":3061,"meta":3062},"pw.io.bigquery","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fbigquery","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F120.bigquery",{"sidebar":2632},{"title":3064,"path":3065,"stem":3066,"meta":3067},"pw.io.clickhouse","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fclickhouse","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F130.clickhouse",{"sidebar":2632},{"title":3069,"path":3070,"stem":3071,"meta":3072},"pw.io.csv","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fcsv","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F140.csv",{"sidebar":2632},{"title":3074,"path":3075,"stem":3076,"meta":3077},"pw.io.debezium","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fdebezium","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F150.debezium",{"sidebar":2632},{"title":3079,"path":3080,"stem":3081,"meta":3082},"pw.io.deltalake","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fdeltalake","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F160.deltalake",{"sidebar":2632},{"title":3084,"path":3085,"stem":3086,"meta":3087},"pw.io.dynamodb","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fdynamodb","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F170.dynamodb",{"sidebar":2632},{"title":3089,"path":3090,"stem":3091,"meta":3092},"pw.io.elasticsearch","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Felasticsearch","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F180.elasticsearch",{"sidebar":2632},{"title":3094,"path":3095,"stem":3096,"meta":3097},"pw.io.fs","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Ffs","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F190.fs",{"sidebar":2632},{"title":3099,"path":3100,"stem":3101,"meta":3102},"pw.io.gdrive","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fgdrive","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F200.gdrive",{"sidebar":2632},{"title":3104,"path":3105,"stem":3106,"meta":3107},"pw.io.http","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fhttp","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F210.http",{"sidebar":2632},{"title":3109,"path":3110,"stem":3111,"meta":3112},"pw.io.iceberg","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Ficeberg","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F220.iceberg",{"sidebar":2632},{"title":3114,"path":3115,"stem":3116,"meta":3117},"pw.io.jsonlines","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fjsonlines","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F230.jsonlines",{"sidebar":2632},{"title":3119,"path":3120,"stem":3121,"meta":3122},"pw.io.kafka","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fkafka","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F240.kafka",{"sidebar":2632},{"title":3124,"path":3125,"stem":3126,"meta":3127},"pw.io.kinesis","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fkinesis","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F250.kinesis",{"sidebar":2632},{"title":3129,"path":3130,"stem":3131,"meta":3132},"pw.io.leann","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fleann","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F260.leann",{"sidebar":2632},{"title":3134,"path":3135,"stem":3136,"meta":3137},"pw.io.logstash","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Flogstash","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F270.logstash",{"sidebar":2632},{"title":3139,"path":3140,"stem":3141,"meta":3142},"pw.io.milvus","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fmilvus","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F280.milvus",{"sidebar":2632},{"title":3144,"path":3145,"stem":3146,"meta":3147},"pw.io.minio","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fminio","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F290.minio",{"sidebar":2632},{"title":3149,"path":3150,"stem":3151,"meta":3152},"pw.io.mongodb","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fmongodb","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F300.mongodb",{"sidebar":2632},{"title":3154,"path":3155,"stem":3156,"meta":3157},"pw.io.mqtt","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fmqtt","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F310.mqtt",{"sidebar":2632},{"title":3159,"path":3160,"stem":3161,"meta":3162},"pw.io.mssql","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fmssql","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F320.mssql",{"sidebar":2632},{"title":3164,"path":3165,"stem":3166,"meta":3167},"pw.io.mysql","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fmysql","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F330.mysql",{"sidebar":2632},{"title":3169,"path":3170,"stem":3171,"meta":3172},"pw.io.nats","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fnats","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F340.nats",{"sidebar":2632},{"title":3174,"path":3175,"stem":3176,"meta":3177},"pw.io.null","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fnull","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F350.null",{"sidebar":2632},{"title":3179,"path":3180,"stem":3181,"meta":3182},"pw.io.plaintext","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fplaintext","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F360.plaintext",{"sidebar":2632},{"title":3184,"path":3185,"stem":3186,"meta":3187},"pw.io.postgres","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fpostgres","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F370.postgres",{"sidebar":2632},{"title":3189,"path":3190,"stem":3191,"meta":3192},"pw.io.pubsub","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fpubsub","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F380.pubsub",{"sidebar":2632},{"title":3194,"path":3195,"stem":3196,"meta":3197},"pw.io.pyfilesystem","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fpyfilesystem","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F390.pyfilesystem",{"sidebar":2632},{"title":3199,"path":3200,"stem":3201,"meta":3202},"pw.io.python","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fpython","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F400.python",{"sidebar":2632},{"title":3204,"path":3205,"stem":3206,"meta":3207},"pw.io.questdb","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fquestdb","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F410.questdb",{"sidebar":2632},{"title":3209,"path":3210,"stem":3211,"meta":3212},"pw.io.rabbitmq","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Frabbitmq","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F420.rabbitmq",{"sidebar":2632},{"title":3214,"path":3215,"stem":3216,"meta":3217},"pw.io.redpanda","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fredpanda","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F430.redpanda",{"sidebar":2632},{"title":3219,"path":3220,"stem":3221,"meta":3222},"pw.io.s3","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fs3","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F440.s3",{"sidebar":2632},{"title":3224,"path":3225,"stem":3226,"meta":3227},"pw.io.slack","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fslack","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F450.slack",{"sidebar":2632},{"title":3229,"path":3230,"stem":3231,"meta":3232},"pw.io.sqlite","\u002Fdevelopers\u002Fapi-docs\u002Fpathway-io\u002Fsqlite","2.developers\u002F5.API-docs\u002F7.pathway-io\u002F460.sqlite",{"sidebar":2632},{"sidebar":2632},{"title":3235,"path":3236,"stem":3237,"meta":3238},"pw.ml","\u002Fdevelopers\u002Fapi-docs\u002Fml","2.developers\u002F5.API-docs\u002F8.ML",{"sidebar":2632},{"title":3240,"path":3241,"stem":3242,"meta":3243},"pw.persistence","\u002Fdevelopers\u002Fapi-docs\u002Fpersistence-api","2.developers\u002F5.API-docs\u002F9.persistence-api",{"sidebar":2632},{"title":3245,"path":3246,"stem":3247,"children":3248,"meta":3990},"Pathway Live Data Framework Templates","\u002Fdevelopers\u002Ftemplates","2.developers\u002F7.templates\u002F1.index",[3249,3251,3256,3261,3266,3270,3291,3330,3356,3788],{"title":3245,"path":3246,"stem":3247,"meta":3250},{"navTitle":2071,"layout":2142,"aside":405,"toc":2075,"single":405},{"title":3252,"path":3253,"stem":3254,"meta":3255},"Run a template","\u002Fdevelopers\u002Ftemplates\u002Frun-a-template","2.developers\u002F7.templates\u002F20.run-a-template",{"heading":2075,"toc":2075},{"title":3257,"path":3258,"stem":3259,"meta":3260},"Customizing a RAG Template with YAML","\u002Fdevelopers\u002Ftemplates\u002Fconfigure-yaml","2.developers\u002F7.templates\u002F30.configure-yaml",{"heading":2075},{"title":3262,"path":3263,"stem":3264,"meta":3265},"How to Use Your Own Components in YAML Configuration","\u002Fdevelopers\u002Ftemplates\u002Fcustom-components","2.developers\u002F7.templates\u002F35.custom-components",{"heading":2075},{"title":2101,"path":3267,"stem":3268,"meta":3269},"\u002Fdevelopers\u002Ftemplates\u002Flicensing-guide","2.developers\u002F7.templates\u002F38.licensing-guide",{"heading":2075,"toc":2075},{"title":3271,"meta":3272,"path":3273,"stem":3274,"children":3275,"page":2075},"YAML Snippets",{},"\u002Fdevelopers\u002Ftemplates\u002Fyaml-snippets","2.developers\u002F7.templates\u002F39.yaml-snippets",[3276,3281,3286],{"title":3277,"path":3278,"stem":3279,"meta":3280},"Data Sources Examples","\u002Fdevelopers\u002Ftemplates\u002Fyaml-snippets\u002Fdata-sources-examples","2.developers\u002F7.templates\u002F39.yaml-snippets\u002F10.data-sources-examples",{"heading":2075,"toc":405},{"title":3282,"path":3283,"stem":3284,"meta":3285},"RAG configuration Examples","\u002Fdevelopers\u002Ftemplates\u002Fyaml-snippets\u002Frag-configuration-examples","2.developers\u002F7.templates\u002F39.yaml-snippets\u002F20.rag-configuration-examples",{"heading":2075,"toc":405},{"title":3287,"path":3288,"stem":3289,"meta":3290},"Full Templates Examples","\u002Fdevelopers\u002Ftemplates\u002Fyaml-snippets\u002Ffull-pipelines-examples","2.developers\u002F7.templates\u002F39.yaml-snippets\u002F30.full-pipelines-examples",{"heading":2075,"toc":405},{"title":3292,"path":3293,"stem":3294,"children":3295,"page":2075},"Rag Customization","\u002Fdevelopers\u002Ftemplates\u002Frag-customization","2.developers\u002F7.templates\u002F40.rag-customization",[3296,3301,3306,3312,3318,3324],{"title":3297,"path":3298,"stem":3299,"meta":3300},"REST API","\u002Fdevelopers\u002Ftemplates\u002Frag-customization\u002Frest-api","2.developers\u002F7.templates\u002F40.rag-customization\u002F10.REST-API",{"heading":2075,"toc":2075},{"title":3302,"path":3303,"stem":3304,"meta":3305},"Customizing prompt","\u002Fdevelopers\u002Ftemplates\u002Frag-customization\u002Fcustom-prompt","2.developers\u002F7.templates\u002F40.rag-customization\u002F20.custom-prompt",{"heading":2075},{"title":2647,"path":3307,"stem":3308,"meta":3309},"\u002Fdevelopers\u002Ftemplates\u002Frag-customization\u002Fparsers","2.developers\u002F7.templates\u002F40.rag-customization\u002F30.parsers",{"date":2651,"tags":3310,"keywords":3311},[2041,2042],[2042,2654,2655,2656,2657],{"title":2659,"path":3313,"stem":3314,"meta":3315},"\u002Fdevelopers\u002Ftemplates\u002Frag-customization\u002Fsplitters","2.developers\u002F7.templates\u002F40.rag-customization\u002F40.splitters",{"date":2663,"thumbnail":130,"tags":3316,"keywords":3317},[2665,2666],[2656,2666],{"title":2669,"path":3319,"stem":3320,"meta":3321},"\u002Fdevelopers\u002Ftemplates\u002Frag-customization\u002Fembedders","2.developers\u002F7.templates\u002F40.rag-customization\u002F50.embedders",{"date":2663,"thumbnail":130,"tags":3322,"keywords":3323},[2041,690],[2042,2045,2046,2595,2675,549],{"title":2677,"path":3325,"stem":3326,"meta":3327},"\u002Fdevelopers\u002Ftemplates\u002Frag-customization\u002Fllm-chats","2.developers\u002F7.templates\u002F40.rag-customization\u002F60.llm-chats",{"date":2680,"thumbnail":130,"tags":3328,"keywords":3329},[2041,2042,2682,2677],[2042,2045,2046,2595,2675,2684],{"title":2695,"path":3331,"stem":3332,"children":3333,"page":2075},"\u002Fdevelopers\u002Ftemplates\u002Fdeploy","2.developers\u002F7.templates\u002F60.deploy",[3334,3338,3342,3349],{"title":2701,"path":3335,"stem":3336,"meta":3337},"\u002Fdevelopers\u002Ftemplates\u002Fdeploy\u002Fcloud-deployment","2.developers\u002F7.templates\u002F60.deploy\u002F10.cloud-deployment",{},{"title":2706,"path":3339,"stem":3340,"meta":3341},"\u002Fdevelopers\u002Ftemplates\u002Fdeploy\u002Fgcp-deploy","2.developers\u002F7.templates\u002F60.deploy\u002F15.gcp-deploy",{},{"title":2711,"path":3343,"stem":3344,"meta":3345},"\u002Fdevelopers\u002Ftemplates\u002Fdeploy\u002Faws-fargate-deploy","2.developers\u002F7.templates\u002F60.deploy\u002F20.aws-fargate-deploy",{"date":2715,"tags":3346,"thumbnail":3347,"keywords":3348,"docker_github_link":2726,"deployButtons":2075},[2717,2718],{"src":2720},[2722,2723,2724,2269,2725],{"title":2728,"path":3350,"stem":3351,"meta":3352},"\u002Fdevelopers\u002Ftemplates\u002Fdeploy\u002Fazure-aci-deploy","2.developers\u002F7.templates\u002F60.deploy\u002F25.azure-aci-deploy",{"date":2732,"tags":3353,"thumbnail":3354,"keywords":3355,"docker_github_link":2742,"deployButtons":2075},[2717,2718],{"src":2735,"contain":405},[2737,2738,2724,2269,2739,2740,2741],{"title":3357,"meta":3358,"path":3359,"stem":3360,"children":3361,"page":2075},"ETL Templates",{},"\u002Fdevelopers\u002Ftemplates\u002Fetl","2.developers\u002F7.templates\u002FETL",[3362,3376,3392,3411,3431,3442,3456,3467,3477,3492,3507,3521,3536,3554,3570,3584,3600,3616,3628,3641,3651,3663,3675,3689],{"title":3363,"path":3364,"stem":3365,"meta":3366},"EL Pipeline: Move your data around with Pathway","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fel-pipeline","2.developers\u002F7.templates\u002FETL\u002F10.el-pipeline",{"layout":3367,"date":3368,"tags":3369,"thumbnail":3370,"keywords":3372,"popular":405},"template","2025-05-22",[2717,2718],{"src":3371,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Fel-template\u002Fel-template-thumbnail.png",[3373,3374,3375,2267,2294,2269],"ETL","EL","YAML",{"title":3377,"path":3378,"stem":3379,"meta":3380},"Real-Time Anomaly Detection: identifying brute-force logins using Tumbling Windows","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fsuspicious_activity_tumbling_window","2.developers\u002F7.templates\u002FETL\u002F10.suspicious_activity_tumbling_window",{"notebook_export_path":3381,"aside":405,"date":3382,"thumbnail":3383,"tags":3386,"keywords":3388,"layout":3367},"notebooks\u002Ftutorials\u002Fsuspicious_user_activity.ipynb","2023-05-30",{"src":3384,"provider":3385},"\u002Fassets\u002Fblog\u002Fthumbnails\u002Fth-shield.png","s3",[2041,3387],"machine-learning",[2567,3389,3390,3391],"tumbling","alert","notebook",{"title":3393,"path":3394,"stem":3395,"meta":3396},"Gaussian Filtering in Real-time: Signal processing with out-of-order data streams","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fgaussian_filtering_python","2.developers\u002F7.templates\u002FETL\u002F100.gaussian_filtering_python",{"notebook_export_path":3397,"aside":405,"date":3398,"layout":3367,"thumbnail":3399,"tags":3401,"related":3403,"keywords":3406},"notebooks\u002Ftutorials\u002Fgaussian-filtering-python.ipynb","2023-10-17",{"src":3400},"\u002Fassets\u002Fcontent\u002Ftutorials\u002Ftime_series\u002Fthumbnail-gaussian.png",[2041,3402],"Time Series",[3404,3405],"\u002Fblog\u002Fsignal-processing","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fupsampling",[3407,3408,2532,3409,3410,2567,3391],"signal processing","Gaussian filter","irregular sampling","intervals_over",{"title":3412,"path":3413,"stem":3414,"meta":3415},"Computing the Option Greeks using Pathway Live Data Framework and Databento","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Foption-greeks","2.developers\u002F7.templates\u002FETL\u002F105.option-greeks",{"thumbnail":3416,"date":3418,"tags":3419,"keywords":3420,"notebook_export_path":3429,"docker_github_link":3430,"aside":405,"layout":3367},{"src":3417,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Foption-greeks\u002Foption-greeks.png","2024-08-06",[2041,2718],[3421,3422,3423,3424,3425,3426,3427,3428,3391],"Option Greeks","Databento","Delta","Gamma","Theta","Rho","quant","Black model","notebooks\u002Ftutorials\u002Foption-greeks.ipynb","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Foption-greeks",{"title":3432,"path":3433,"stem":3434,"meta":3435},"Automating reconciliation of messy financial transaction logs using the Pathway Live Data Framework real-time fuzzy join","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Ffuzzy_join_chapter1","2.developers\u002F7.templates\u002FETL\u002F11.fuzzy_join_chapter1",{"notebook_export_path":3436,"aside":405,"date":3437,"tags":3438,"keywords":3439,"layout":3367},"notebooks\u002Fshowcases\u002Ffuzzy_join_part1.ipynb","2022-10-18",[2041,2718],[3440,3441,2654,3391],"Fuzzy join","reconciliation",{"title":3443,"path":3405,"stem":3444,"meta":3445},"Signal Processing with Real-time Upsampling: combining multiple time series data streams.","2.developers\u002F7.templates\u002FETL\u002F110.upsampling",{"notebook_export_path":3446,"aside":405,"date":3447,"layout":3367,"thumbnail":3448,"tags":3450,"related":3451,"popular":405,"keywords":3452},"notebooks\u002Ftutorials\u002Fupsampling.ipynb","2023-10-18",{"src":3449},"\u002Fassets\u002Fcontent\u002Ftutorials\u002Ftime_series\u002Fthumbnail-time-series.png",[2041,3402],[3404,3394],[3407,3453,2532,3454,3455,3410,2567,3391],"upsampling","resampling","interpolation",{"title":3457,"path":3458,"stem":3459,"meta":3460},"Interaction with a Feedback Loop.","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Ffuzzy_join_chapter2","2.developers\u002F7.templates\u002FETL\u002F12.fuzzy_join_chapter2",{"notebook_export_path":3461,"aside":405,"date":3462,"thumbnail":3463,"tags":3465,"keywords":3466,"layout":3367},"notebooks\u002Fshowcases\u002Ffuzzy_join_part2.ipynb","2022-10-19",{"src":3464,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Ffuzzy_join\u002Freconciliation_chapter3_trim.png",[2041,2718],[3440,3441,2654,3391],{"title":3468,"path":3469,"stem":3470,"meta":3471},"Smart real-time monitoring application with alert deduplication","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Falerting-significant-changes","2.developers\u002F7.templates\u002FETL\u002F130.alerting-significant-changes",{"notebook_export_path":3472,"aside":405,"date":3473,"tags":3474,"keywords":3475,"layout":3367},"notebooks\u002Ftutorials\u002Falert-deduplication.ipynb","2023-11-16",[2041,2718],[3390,3476,2784,3391],"deduplication",{"title":3478,"path":3479,"stem":3480,"meta":3481},"Kafka ETL: Processing event streams in Python","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fkafka-etl","2.developers\u002F7.templates\u002FETL\u002F140.kafka-etl",{"layout":3367,"date":3482,"tags":3483,"thumbnail":3484,"keywords":3486,"docker_github_link":3491,"popular":405},"2024-02-15",[2717,2718],{"src":3485,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002FETL-Kafka\u002FETL-Kafka.png",[3373,2267,3487,3488,3489,3490,2269],"datetime","time zone","timezone","concat_reindex","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fkafka-ETL",{"title":3493,"path":3494,"stem":3495,"meta":3496},"Streaming ETL pipelines in Python with Airbyte and Pathway","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fetl-python-airbyte","2.developers\u002F7.templates\u002FETL\u002F150.etl-python-airbyte",{"layout":3367,"date":3497,"tags":3498,"thumbnail":3499,"keywords":3501},"2024-02-28",[2717,2718],{"src":3500},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Fairbyte\u002Fairbyte-diagram-th.png",[3502,3503,3504,3505,3506],"airbyte","processing","airbyte python","python etl","airbyte etl",{"title":3508,"path":3509,"stem":3510,"meta":3511},"Delta Lake ETL with Pathway Live Data Framework for Spark Analytics","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fdelta_lake_etl","2.developers\u002F7.templates\u002FETL\u002F175.delta_lake_etl",{"layout":3367,"date":3512,"tags":3513,"thumbnail":3514,"keywords":3516,"docker_github_link":3520},"2024-07-23",[2717,2718],{"src":3515,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Fdeltalake\u002Fdelta_lake_diagram_th.png",[3517,3518,3519,3505,3506,2269],"delta lake etl","spark delta lake","spark etl","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fspark-data-preparation",{"title":3522,"path":3523,"stem":3524,"meta":3525},"Python Kafka Alternative: Achieve Sub-Second Latency with your S3 Storage without Kafka using Pathway","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fkafka-alternative","2.developers\u002F7.templates\u002FETL\u002F180.kafka-alternative",{"layout":3367,"thumbnail":3526,"date":3528,"tags":3529,"keywords":3530},{"src":3527,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Fkafka-alternatives\u002Fkafka-alternatives-thumbnail.png","2024-08-27",[2717,2718],[3373,2267,3531,3532,3533,3489,3534,3535],"Pathway","MinIO","S3","Kafka-alternatives","Delta tables",{"title":3537,"path":3538,"stem":3539,"meta":3540},"Jupyter \u002F Colab: visualizing and transforming live data streams in Python notebooks with Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Flive_data_jupyter","2.developers\u002F7.templates\u002FETL\u002F2.live_data_jupyter",{"aside":405,"date":2763,"tags":3541,"keywords":3542,"notebook_export_path":3552,"docker_github_link":3553,"popular":405,"layout":3367},[2041,2718,2717],[3543,3544,3545,3546,3547,3548,3549,3550,3551,3391,2269],"Jupyter notebook","interactivity","bokeh","panel","Bollinger bands","vizualization","Weighted Average","rolling statistics","dashboard","notebooks\u002Fshowcases\u002Flive-data-jupyter.ipynb","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Ffrom_jupyter_to_deploy\u002Fpart4_deployment",{"title":3555,"path":3556,"stem":3557,"meta":3558},"Real-Time Twitter Sentiment Analysis and Prediction App with Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Ftwitter","2.developers\u002F7.templates\u002FETL\u002F2.twitter",{"aside":405,"layout":3367,"date":3559,"thumbnail":3560,"tags":3562,"keywords":3563,"docker_github_link":3569},"2022-10-31",{"src":3561},"\u002Fassets\u002Fcontent\u002Fblog\u002Fth-twitter.png",[2041,3387,2717],[3564,3565,3566,3567,3568,2654,2269],"Twitter","tweets","sentiment analysis","geolocation","influence","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Ftwitter",{"title":3571,"path":3572,"stem":3573,"meta":3574},"Out-of-Order Event Streams: Calculating Time Deltas with grouping by topic","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fevent_stream_processing_time_between_occurrences","2.developers\u002F7.templates\u002FETL\u002F20.event_stream_processing_time_between_occurrences",{"aside":405,"date":3575,"layout":3367,"thumbnail":3576,"tags":3578,"keywords":3579},"2022-11-01",{"src":3577},"\u002Fassets\u002Fcontent\u002Fblog\u002Fth-time-between-events-in-a-multi-topic-event-stream.png",[2041,2718],[3580,3581,2266,3582,3583],"event stream","multi-topic","ordering","sort",{"title":3585,"path":3586,"stem":3587,"meta":3588},"Adaptive Classifiers: Evolving Predictions with Real-Time Data","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Flsh_chapter1","2.developers\u002F7.templates\u002FETL\u002F3.lsh_chapter1",{"notebook_export_path":3589,"aside":405,"date":3590,"thumbnail":3591,"tags":3593,"keywords":3594,"layout":3367},"notebooks\u002Fshowcases\u002Flsh.ipynb","2022-10-25",{"src":3592},"\u002Fassets\u002Fcontent\u002Fblog\u002Fth-realtime-classification.png",[2041,3387],[3595,2465,3596,2462,3597,3598,3599,2654,3391],"Classification","LSH","Locality Sensitive Hashing","MNIST","euclidean distance",{"title":3601,"path":3602,"stem":3603,"meta":3604},"Uncovering hidden user relationships in crypto exchanges with Fuzzy Join on streaming data","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fuser_pairs_fuzzy_join","2.developers\u002F7.templates\u002FETL\u002F30.user_pairs_fuzzy_join",{"aside":405,"date":3605,"thumbnail":3606,"tags":3608,"keywords":3609,"layout":3367},"2023-01-09",{"src":3607,"provider":3385},"\u002Fassets\u002Fblog\u002Fthumbnails\u002Fth-mining-hidden-user-pair-activity-with-fuzzy-join.png",[2041,2718],[3610,3390,3611,3612,3613,3614,3615],"fuzzy join","cryptocurrency","bitcoin","BTC","ETH","Ethereum",{"title":3617,"path":3618,"stem":3619,"meta":3620},"Pathway Live Data Framework Logistics Application: Streamlined Insights for Real-Time Asset Management","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Flogistics","2.developers\u002F7.templates\u002FETL\u002F4.logistics",{"aside":405,"layout":3367,"date":3559,"thumbnail":3621,"tags":3623,"enterprise":405,"keywords":3624},{"src":3622},"\u002Fassets\u002Fcontent\u002Fblog\u002Fth-logictics-app.png",[2041,3387],[3625,3626,3627],"Logistics","IoT","Dashboard",{"title":3629,"path":3630,"stem":3631,"meta":3632},"Real-Time Shortest Paths on Dynamic Networks with Bellman-Ford in Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fbellman_ford","2.developers\u002F7.templates\u002FETL\u002F40.bellman_ford",{"aside":405,"date":3633,"layout":3367,"thumbnail":3634,"tags":3636,"notebook_export_path":3637,"keywords":3638},"2022-11-03",{"src":3635,"provider":3385},"\u002Fassets\u002Fblog\u002Fthumbnails\u002Fth-bellman-ford.png",[2041,3387],"notebooks\u002Ftutorials\u002Fbellman_ford.ipynb",[2443,3639,3640,2798,3391],"fixed-point","shortest-path",{"title":3642,"path":3643,"stem":3644,"meta":3645},"Linear regression on a Kafka stream","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Flinear_regression_with_kafka","2.developers\u002F7.templates\u002FETL\u002F5.linear_regression_with_kafka",{"aside":405,"layout":3367,"date":3646,"tags":3647,"keywords":3648,"docker_github_link":3650},"2022-12-23",[2041,2718],[3649,2267,2199],"linear regression","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Fkafka-linear-regression",{"title":3652,"path":3653,"stem":3654,"meta":3655},"Real-Time PageRank on Dynamic Graphs with Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fpagerank","2.developers\u002F7.templates\u002FETL\u002F50.pagerank",{"notebook_export_path":3656,"aside":405,"date":3657,"thumbnail":3658,"tags":3660,"keywords":3661,"layout":3367},"notebooks\u002Ftutorials\u002Fpagerank.ipynb","2022-11-07",{"src":3659,"provider":3385},"\u002Fassets\u002Fblog\u002Fthumbnails\u002Fth-computing-pagerank.png",[2041,3387],[3662,2798,3391],"pagerank",{"title":3664,"path":3665,"stem":3666,"meta":3667},"Sensor Fusion in real-time: combining time series data with Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Fcombining_time_series","2.developers\u002F7.templates\u002FETL\u002F65.combining_time_series",{"aside":405,"date":3668,"thumbnail":3669,"tags":3670,"keywords":3671,"notebook_export_path":3674,"layout":3367},"2023-04-28",{"src":3449},[3402],[3672,3673,3455,2163,3391],"time series","multiple data sources","notebooks\u002Ftutorials\u002Fcombining-time-series.ipynb",{"title":3676,"path":3677,"stem":3678,"meta":3679},"Realtime Server Log Monitoring: nginx + Filebeat + Pathway","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002Frealtime-log-monitoring","2.developers\u002F7.templates\u002FETL\u002F7.realtime-log-monitoring",{"aside":405,"layout":3367,"date":3680,"thumbnail":3681,"tags":3683,"keywords":3684,"docker_github_link":3688},"2023-02-27",{"src":3682,"contain":405},"\u002Fassets\u002Fcontent\u002Ftutorials\u002Frealtime_log_monitoring\u002Fmeme.jpg",[2041,2718],[3685,3390,2267,3686,2242,3687,2269],"log monitoring","Filebeat","ElasticSearch","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fpathway\u002Ftree\u002Fmain\u002Fexamples\u002Fprojects\u002Frealtime-log-monitoring\u002Ffilebeat-pathway-slack",{"title":3690,"path":3691,"stem":3692,"children":3693,"page":2075},"Readmes","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes","2.developers\u002F7.templates\u002FETL\u002F_readmes",[3694,3699,3704,3709,3714,3719,3724,3729,3734,3739,3744,3748,3753,3758,3763,3768,3773,3778,3783],{"title":3695,"path":3696,"stem":3697,"meta":3698},"AG2 Multi-Agent Conversations with Pathway Live Data Framework Real-Time RAG","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fag2-multiagent-rag","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fag2-multiagent-rag",{},{"title":3700,"path":3701,"stem":3702,"meta":3703},"Azure ACI Deployment Example","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fazure-aci-deploy","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fazure-aci-deploy",{},{"title":3705,"path":3706,"stem":3707,"meta":3708},"Benchmark for Delta Lake S3 messaging as a Kafka replacement","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fbenchmarks","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fbenchmarks",{},{"title":3710,"path":3711,"stem":3712,"meta":3713},"Best-rated movies examples","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fbest-movies-example","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fbest-movies-example",{},{"title":3715,"path":3716,"stem":3717,"meta":3718},"Make your LLM app sane again: Forgetting incorrect data in real time","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fconf42","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fconf42",{},{"title":3720,"path":3721,"stem":3722,"meta":3723},"Custom python connector example","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fcustom-python-connector-twitter","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fcustom-python-connector-twitter",{},{"title":3725,"path":3726,"stem":3727,"meta":3728},"Realtime monitoring of logs","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Ffilebeat-pathway-slack","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Ffilebeat-pathway-slack",{},{"title":3730,"path":3731,"stem":3732,"meta":3733},"Tutorial: From interactive data exploration to deployment","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Ffrom_jupyter_to_deploy","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Ffrom_jupyter_to_deploy",{},{"title":3735,"path":3736,"stem":3737,"meta":3738},"ETL with Kafka in\u002FKafka out","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fkafka-etl","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fkafka-ETL",{},{"title":3740,"path":3741,"stem":3742,"meta":3743},"Best-rated movies example - Kafka version","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fkafka-version","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fkafka-version",{},{"title":3725,"path":3745,"stem":3746,"meta":3747},"\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Flogstash-pathway-elastic","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Flogstash-pathway-elastic",{},{"title":3749,"path":3750,"stem":3751,"meta":3752},"Pathway Live Data Framework Monitoring using OpenTelemetry Collector and Grafana Cloud","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fmonitoring","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fmonitoring",{},{"title":3754,"path":3755,"stem":3756,"meta":3757},"Computing Option Greeks with Pathway Live Data Framework and Databento.","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Foption-greeks","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Foption-greeks",{},{"title":3759,"path":3760,"stem":3761,"meta":3762},"Retrieval-Augmented Generation (RAG) Pipeline with Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fquestion-answering-rag","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fquestion-answering-rag",{},{"title":3764,"path":3765,"stem":3766,"meta":3767},"Best-rated movies example - Redpanda version","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fredpanda-version","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fredpanda-version",{},{"title":3769,"path":3770,"stem":3771,"meta":3772},"Sample Pathway Live Data Framework program for SharePoint connection testing","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fsharepoint-test","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fsharepoint-test",{},{"title":3774,"path":3775,"stem":3776,"meta":3777},"Data Preparation for Spark Analytics","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fspark-data-preparation","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fspark-data-preparation",{},{"title":3779,"path":3780,"stem":3781,"meta":3782},"Realtime Twitter Analysis App with Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Ftwitter","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Ftwitter",{},{"title":3784,"path":3785,"stem":3786,"meta":3787},"Web Scraping with Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Fetl\u002F_readmes\u002Fweb-scraping","2.developers\u002F7.templates\u002FETL\u002F_readmes\u002Fweb-scraping",{},{"title":3789,"meta":3790,"path":3791,"stem":3792,"children":3793,"page":2075},"LLM-RAG Templates",{},"\u002Fdevelopers\u002Ftemplates\u002Frag","2.developers\u002F7.templates\u002Frag",[3794,3807,3822,3838,3851,3869,3879,3894,3904,3914,3925,3938],{"title":3795,"path":3796,"stem":3797,"meta":3798},"Question-Answering RAG App","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Fdemo-question-answering","2.developers\u002F7.templates\u002Frag\u002F1000.demo-question-answering",{"redirection":3799,"thumbnail":3800,"layout":3367,"tags":3802,"date":2651,"keywords":3804,"docker_github_link":3799,"popular":405},"https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Fquestion_answering_rag",{"src":3801,"provider":3385},"\u002Fassets\u002Fblog\u002Fthumbnails\u002Fqna-th.png",[2717,3803],"ai-pipelines",[2042,2043,2045,2046,3805,230,2199,3806],"vector store","yaml",{"title":3808,"path":3809,"stem":3810,"meta":3811},"Adaptive RAG App","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Ftemplate-adaptive-rag","2.developers\u002F7.templates\u002Frag\u002F1001.template-adaptive-rag",{"thumbnail":3812,"tags":3814,"date":3815,"layout":3367,"keywords":3816,"docker_github_link":3821},{"src":3813,"contain":405},"\u002Fassets\u002Fcontent\u002Fblog\u002Fadaptive-rag-plots\u002Fvisual-abstract.png",[2717,3803],"2024-03-29",[2042,2043,3817,3818,3819,3820,2199,3806],"Adaptive RAG","prompt engineering","prompt","explainability","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Fadaptive_rag",{"title":3823,"path":3824,"stem":3825,"meta":3826},"Private RAG App with Mistral and Ollama","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Ftemplate-private-rag","2.developers\u002F7.templates\u002Frag\u002F1002.template-private-rag",{"tags":3827,"date":3828,"thumbnail":3829,"layout":3367,"keywords":3831,"docker_github_link":3837},[2717,3803],"2024-04-22",{"src":3830,"contain":405},"\u002Fassets\u002Fcontent\u002Fblog\u002Flocal-adaptive-rag\u002Flocal_adaptive.png",[2042,2043,3817,3818,3820,3832,3833,3834,3835,3836,2199,3806],"mistral","ollama","private rag","local rag","ollama rag","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Fprivate_rag",{"title":3839,"path":3840,"stem":3841,"meta":3842},"Multimodal RAG pipeline with GPT4o","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Ftemplate-multimodal-rag","2.developers\u002F7.templates\u002Frag\u002F1003.template-multimodal-rag",{"tags":3843,"date":3844,"thumbnail":3845,"layout":3367,"keywords":3847,"docker_github_link":3850,"popular":405},[2717,3803],"2024-01-01",{"src":3846,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Fmultimodal-RAG\u002FmultimodalRAG-blog-banner.png",[2042,2043,2045,2046,3848,3849,2654,2199,3806],"GPT-4o","multimodal RAG","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Fmultimodal_rag",{"title":3852,"path":3853,"stem":3854,"children":3855,"meta":3865},"Live Document Indexing (Vector Store \u002F Retriever)","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Ftemplate-demo-document-indexing","2.developers\u002F7.templates\u002Frag\u002F1008.template-demo-document-indexing",[3856],{"title":3852,"path":3853,"stem":3854,"meta":3857},{"thumbnail":3858,"tags":3860,"date":3844,"layout":3367,"keywords":3861,"docker_github_link":3864},{"src":3859,"provider":3385},"\u002Fassets\u002Fblog\u002Fthumbnails\u002Flive-document-indexing-th.png",[2717,3803],[2042,2043,2045,2046,3805,230,3862,2360,3863,2199,3806],"SharePoint","Gdrive","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Fdocument_indexing",{"thumbnail":3866,"tags":3867,"date":3844,"layout":3367,"keywords":3868,"docker_github_link":3864},{"src":3859,"provider":3385},[2717,3803],[2042,2043,2045,2046,3805,230,3862,2360,3863,2199,3806],{"title":3870,"path":3871,"stem":3872,"meta":3873},"Alerting when answers change on Google Drive","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Fdrive-alert","2.developers\u002F7.templates\u002Frag\u002F1009.drive-alert",{"tags":3874,"date":3875,"layout":3367,"keywords":3876,"docker_github_link":3878,"hide":405},[2717,3803],"2024-11-07",[2042,2043,2045,2046,3877,230,2360,3863,2199],"slack","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Fdrive_alert",{"title":3880,"path":3881,"stem":3882,"meta":3883},"Slides AI Search App","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Ftemplate-slides-search","2.developers\u002F7.templates\u002Frag\u002F1010.template-slides-search",{"thumbnail":3884,"tags":3886,"date":3887,"layout":3367,"keywords":3888,"docker_github_link":3893},{"src":3885,"provider":3385},"\u002Fassets\u002Fblog\u002Fthumbnails\u002Fslides-search-th.png",[2717,3803],"2024-11-14",[2042,2043,2045,2046,2360,3889,2199,3806,3890,3891,3892],"slides","multimodal","VLM","image-to-text","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Fslides_ai_search",{"title":3895,"path":3896,"stem":3897,"meta":3898},"Adaptive RAG: cut your LLM costs without sacrificing accuracy","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Fadaptive-rag","2.developers\u002F7.templates\u002Frag\u002F1015.adaptive-rag",{"aside":405,"thumbnail":3899,"tags":3900,"date":3901,"notebook_export_path":3902,"keywords":3903,"run_template":3809,"hide":405},{"src":3813,"contain":405},[2717,3803],"2024-03-28","notebooks\u002Fshowcases\u002Fadaptive_rag_question_answering.ipynb",[2042,2043,3817,3818,3819,3820,3391,2269],{"title":3905,"path":3906,"stem":3907,"meta":3908},"Multimodal RAG for PDFs with Text, Images, and Charts","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Fmultimodal-rag","2.developers\u002F7.templates\u002Frag\u002F120.multimodal-rag",{"aside":405,"thumbnail":3909,"date":3910,"tags":3911,"keywords":3912,"notebook_export_path":3913,"run_template":3840,"popular":405,"hide":405},{"src":3846,"contain":405},"2024-06-20",[2717,3803],[2042,2043,2045,2046,3848,3849,2654,3391,2199],"notebooks\u002Fshowcases\u002Fmultimodal-rag.ipynb",{"title":3915,"path":3916,"stem":3917,"meta":3918},"Private RAG with Connected Data Sources using Mistral, Ollama, and Pathway Live Data Framework","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Fprivate-rag-ollama-mistral","2.developers\u002F7.templates\u002Frag\u002F161.private-rag-ollama-mistral",{"aside":405,"thumbnail":3919,"tags":3920,"date":3921,"related":3922,"notebook_export_path":3923,"keywords":3924,"run_template":3824,"hide":405},{"src":3830,"contain":405},[2717,3803],"2024-04-23",[3896,3796],"notebooks\u002Fshowcases\u002Fmistral_adaptive_rag_question_answering.ipynb",[2042,2043,3817,3818,3820,3832,3833,3834,3835,3836,3391,2199],{"title":3926,"path":3927,"stem":3928,"meta":3929},"Pathway Live Data Framework + PostgreSQL + LLM: app for querying financial reports with live document structuring pipeline.","\u002Fdevelopers\u002Ftemplates\u002Frag\u002Funstructured-to-structured","2.developers\u002F7.templates\u002Frag\u002F5.unstructured-to-structured",{"aside":405,"layout":3367,"date":2577,"thumbnail":3930,"tags":3932,"keywords":3933,"docker_github_link":3937},{"src":3931,"contain":405},"\u002Fassets\u002Fcontent\u002Fshowcases\u002Fllm-app\u002Farchitecture_unst_to_st.png",[2717,3803],[2042,3934,2654,2294,2045,3935,3936,2162,2654,2269],"SQL","Entity extraction","Document parsing","https:\u002F\u002Fgithub.com\u002Fpathwaycom\u002Fllm-app\u002Ftree\u002Fmain\u002Ftemplates\u002Funstructured_to_sql_on_the_fly",{"title":3690,"path":3939,"stem":3940,"children":3941,"page":2075},"\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes","2.developers\u002F7.templates\u002Frag\u002F_readmes",[3942,3947,3955,3960,3965,3970,3975,3980,3985],{"title":3943,"path":3944,"stem":3945,"meta":3946},"AdaptiveRag","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fadaptive_rag","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fadaptive_rag",{},{"title":3948,"path":3949,"stem":3950,"children":3951,"meta":3954},"DocumentIndexing","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fdocument_indexing","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fdocument_indexing",[3952],{"title":3948,"path":3949,"stem":3950,"meta":3953},{},{},{"title":3956,"path":3957,"stem":3958,"meta":3959},"DocumentStoreMcpServer","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fdocument_store_mcp_server","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fdocument_store_mcp_server",{},{"title":3961,"path":3962,"stem":3963,"meta":3964},"Pathway Live Data Framework + LLM + Slack notification: RAG App with real-time alerting when answers change in documents","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fdrive_alert","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fdrive_alert",{},{"title":3966,"path":3967,"stem":3968,"meta":3969},"MultimodalRag","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fmultimodal_rag","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fmultimodal_rag",{},{"title":3971,"path":3972,"stem":3973,"meta":3974},"PrivateRag","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fprivate_rag","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fprivate_rag",{},{"title":3976,"path":3977,"stem":3978,"meta":3979},"QuestionAnsweringRag","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fquestion_answering_rag","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fquestion_answering_rag",{},{"title":3981,"path":3982,"stem":3983,"meta":3984},"SlidesAiSearch","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Fslides_ai_search","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Fslides_ai_search",{},{"title":3986,"path":3987,"stem":3988,"meta":3989},"UnstructuredToSqlOnTheFly","\u002Fdevelopers\u002Ftemplates\u002Frag\u002F_readmes\u002Funstructured_to_sql_on_the_fly","2.developers\u002F7.templates\u002Frag\u002F_readmes\u002Funstructured_to_sql_on_the_fly",{},{"navTitle":2071,"layout":2142,"aside":405,"toc":2075,"single":405},{"id":4,"title":5,"author":6,"body":3992,"description":2036,"extension":2037,"meta":5544,"navigation":405,"path":2047,"seo":5547,"sitemap":5548,"stem":2050,"__hash__":2051},{"type":8,"value":3993,"toc":5526},[3994,3996,3998,4000,4004,4007,4009,4011,4055,4057,4059,4061,4075,4079,4083,4088,4090,4094,4236,4244,4274,4276,4280,4286,4290,4340,4344,4354,4356,4374,4380,4546,4548,4606,4610,4612,4614,4618,4664,4670,4698,4702,4772,4774,4776,4802,4804,4870,4880,4882,4884,4906,4910,4914,4948,4950,4966,4968,4970,4972,4974,4976,4980,5060,5068,5072,5230,5232,5240,5300,5304,5306,5316,5426,5428,5430,5444,5446,5448,5452,5466,5468,5478,5480,5498,5500,5502,5504,5506,5508,5510,5512,5522,5524],[11,3995,14],{"id":13},[16,3997,18],{},[16,3999,21],{},[16,4001,24,4002,30],{},[26,4003,29],{"href":28},[32,4005],{":zoomable":34,"alt":35,"className":4006,"src":38},[37],[40,4008,43],{"id":42},[16,4010,46],{},[48,4012,4013,4019,4025,4031,4037,4043,4049],{},[51,4014,4015,60],{},[26,4016,4017],{"href":55},[57,4018,59],{},[51,4020,4021,69],{},[26,4022,4023],{"href":65},[57,4024,68],{},[51,4026,4027,78],{},[26,4028,4029],{"href":74},[57,4030,77],{},[51,4032,4033,87],{},[26,4034,4035],{"href":83},[57,4036,86],{},[51,4038,4039,95],{},[26,4040,4041],{"href":83},[57,4042,94],{},[51,4044,4045,104],{},[26,4046,4047],{"href":100},[57,4048,103],{},[51,4050,4051,113],{},[26,4052,4053],{"href":109},[57,4054,112],{},[16,4056,116],{},[40,4058,120],{"id":119},[16,4060,123],{},[125,4062,4063],{"className":127,"code":128,"language":129,"meta":130,"style":130},[132,4064,4065],{"__ignoreMap":130},[135,4066,4067,4069,4071,4073],{"class":137,"line":138},[135,4068,142],{"class":141},[135,4070,146],{"class":145},[135,4072,149],{"class":145},[135,4074,152],{"class":145},[16,4076,155,4077,30],{},[26,4078,159],{"href":158},[16,4080,162,4081,166],{},[132,4082,165],{},[125,4084,4086],{"className":4085,"code":171,"language":172},[170],[132,4087,171],{"__ignoreMap":130},[176,4089,179],{"id":178},[16,4091,182,4092,186],{},[132,4093,185],{},[125,4095,4096],{"className":189,"code":190,"filename":191,"language":192,"meta":130,"style":130},[132,4097,4098,4108,4130,4148,4170,4192,4214],{"__ignoreMap":130},[135,4099,4100,4102,4104,4106],{"class":137,"line":138},[135,4101,200],{"class":199},[135,4103,204],{"class":203},[135,4105,207],{"class":199},[135,4107,210],{"class":203},[135,4109,4110,4112,4114,4116,4118,4120,4122,4124,4126,4128],{"class":137,"line":213},[135,4111,216],{"class":199},[135,4113,219],{"class":203},[135,4115,30],{"class":222},[135,4117,225],{"class":203},[135,4119,30],{"class":222},[135,4121,230],{"class":203},[135,4123,30],{"class":222},[135,4125,235],{"class":203},[135,4127,200],{"class":199},[135,4129,240],{"class":203},[135,4131,4132,4134,4136,4138,4140,4142,4144,4146],{"class":137,"line":243},[135,4133,216],{"class":199},[135,4135,219],{"class":203},[135,4137,30],{"class":222},[135,4139,252],{"class":203},[135,4141,30],{"class":222},[135,4143,257],{"class":203},[135,4145,200],{"class":199},[135,4147,262],{"class":203},[135,4149,4150,4152,4154,4156,4158,4160,4162,4164,4166,4168],{"class":137,"line":265},[135,4151,216],{"class":199},[135,4153,219],{"class":203},[135,4155,30],{"class":222},[135,4157,252],{"class":203},[135,4159,30],{"class":222},[135,4161,278],{"class":203},[135,4163,30],{"class":222},[135,4165,283],{"class":203},[135,4167,200],{"class":199},[135,4169,288],{"class":203},[135,4171,4172,4174,4176,4178,4180,4182,4184,4186,4188,4190],{"class":137,"line":291},[135,4173,216],{"class":199},[135,4175,219],{"class":203},[135,4177,30],{"class":222},[135,4179,252],{"class":203},[135,4181,30],{"class":222},[135,4183,278],{"class":203},[135,4185,30],{"class":222},[135,4187,308],{"class":203},[135,4189,200],{"class":199},[135,4191,313],{"class":203},[135,4193,4194,4196,4198,4200,4202,4204,4206,4208,4210,4212],{"class":137,"line":316},[135,4195,216],{"class":199},[135,4197,219],{"class":203},[135,4199,30],{"class":222},[135,4201,252],{"class":203},[135,4203,30],{"class":222},[135,4205,278],{"class":203},[135,4207,30],{"class":222},[135,4209,333],{"class":203},[135,4211,200],{"class":199},[135,4213,338],{"class":203},[135,4215,4216,4218,4220,4222,4224,4226,4228,4230,4232,4234],{"class":137,"line":341},[135,4217,216],{"class":199},[135,4219,219],{"class":203},[135,4221,30],{"class":222},[135,4223,252],{"class":203},[135,4225,30],{"class":222},[135,4227,278],{"class":203},[135,4229,30],{"class":222},[135,4231,358],{"class":203},[135,4233,200],{"class":199},[135,4235,363],{"class":203},[16,4237,366,4238,370,4240,374,4242,166],{},[132,4239,369],{},[132,4241,373],{},[132,4243,165],{},[125,4245,4246],{"className":189,"code":379,"filename":191,"language":192,"meta":130,"style":130},[132,4247,4248,4258,4264,4268],{"__ignoreMap":130},[135,4249,4250,4252,4254,4256],{"class":137,"line":138},[135,4251,216],{"class":199},[135,4253,388],{"class":203},[135,4255,200],{"class":199},[135,4257,393],{"class":203},[135,4259,4260,4262],{"class":137,"line":213},[135,4261,200],{"class":199},[135,4263,400],{"class":203},[135,4265,4266],{"class":137,"line":243},[135,4267,406],{"emptyLinePlaceholder":405},[135,4269,4270,4272],{"class":137,"line":265},[135,4271,412],{"class":411},[135,4273,415],{"class":222},[40,4275,59],{"id":418},[16,4277,421,4278,30],{},[26,4279,425],{"href":424},[16,4281,428,4282,432,4284,436],{},[57,4283,431],{},[132,4285,435],{},[16,4287,439,4288,30],{},[26,4289,443],{"href":442},[125,4291,4292],{"className":189,"code":446,"filename":191,"language":192,"meta":130,"style":130},[132,4293,4294],{"__ignoreMap":130},[135,4295,4296,4298,4300,4302,4304,4306,4308,4310,4312,4314,4316,4318,4320,4322,4324,4326,4328,4330,4332,4334,4336,4338],{"class":137,"line":138},[135,4297,453],{"class":203},[135,4299,456],{"class":222},[135,4301,459],{"class":203},[135,4303,30],{"class":222},[135,4305,465],{"class":464},[135,4307,30],{"class":222},[135,4309,470],{"class":464},[135,4311,30],{"class":222},[135,4313,475],{"class":411},[135,4315,478],{"class":222},[135,4317,481],{"class":222},[135,4319,435],{"class":145},[135,4321,481],{"class":222},[135,4323,488],{"class":222},[135,4325,492],{"class":491},[135,4327,456],{"class":222},[135,4329,481],{"class":222},[135,4331,499],{"class":145},[135,4333,481],{"class":222},[135,4335,488],{"class":222},[135,4337,506],{"class":491},[135,4339,509],{"class":222},[16,4341,512,4342,30],{},[26,4343,516],{"href":515},[16,4345,519,4346,524,4348,527,4350,30],{},[26,4347,523],{"href":522},[132,4349,523],{},[26,4351,4352],{"href":530},[132,4353,533],{},[16,4355,536],{},[48,4357,4358,4362,4366,4370],{},[51,4359,4360,544],{},[57,4361,543],{},[51,4363,4364,550],{},[57,4365,549],{},[51,4367,4368,556],{},[57,4369,555],{},[51,4371,4372,562],{},[57,4373,561],{},[16,4375,565,4376,572],{},[26,4377,4378],{"href":568},[132,4379,571],{},[125,4381,4382],{"className":189,"code":575,"filename":191,"language":192,"meta":130,"style":130},[132,4383,4384,4394,4422,4426,4456,4466,4476,4480,4490,4504,4510,4524,4538,4542],{"__ignoreMap":130},[135,4385,4386,4388,4390,4392],{"class":137,"line":138},[135,4387,582],{"class":203},[135,4389,456],{"class":222},[135,4391,587],{"class":411},[135,4393,590],{"class":222},[135,4395,4396,4398,4400,4402,4404,4406,4408,4410,4412,4414,4416,4418,4420],{"class":137,"line":213},[135,4397,595],{"class":491},[135,4399,456],{"class":222},[135,4401,601],{"class":600},[135,4403,488],{"class":222},[135,4405,606],{"class":491},[135,4407,456],{"class":222},[135,4409,611],{"class":600},[135,4411,488],{"class":222},[135,4413,616],{"class":491},[135,4415,456],{"class":222},[135,4417,481],{"class":222},[135,4419,623],{"class":145},[135,4421,626],{"class":222},[135,4423,4424],{"class":137,"line":243},[135,4425,631],{"class":222},[135,4427,4428,4430,4432,4434,4436,4438,4440,4442,4444,4446,4448,4450,4452,4454],{"class":137,"line":265},[135,4429,636],{"class":203},[135,4431,456],{"class":222},[135,4433,641],{"class":411},[135,4435,478],{"class":222},[135,4437,646],{"class":491},[135,4439,456],{"class":222},[135,4441,369],{"class":411},[135,4443,30],{"class":222},[135,4445,655],{"class":464},[135,4447,658],{"class":222},[135,4449,481],{"class":222},[135,4451,663],{"class":145},[135,4453,481],{"class":222},[135,4455,668],{"class":222},[135,4457,4458,4460,4462,4464],{"class":137,"line":291},[135,4459,673],{"class":203},[135,4461,456],{"class":222},[135,4463,678],{"class":411},[135,4465,590],{"class":222},[135,4467,4468,4470,4472,4474],{"class":137,"line":316},[135,4469,685],{"class":491},[135,4471,456],{"class":222},[135,4473,690],{"class":411},[135,4475,693],{"class":222},[135,4477,4478],{"class":137,"line":341},[135,4479,631],{"class":222},[135,4481,4482,4484,4486,4488],{"class":137,"line":700},[135,4483,703],{"class":203},[135,4485,456],{"class":222},[135,4487,708],{"class":411},[135,4489,590],{"class":222},[135,4491,4492,4494,4496,4498,4500,4502],{"class":137,"line":713},[135,4493,716],{"class":491},[135,4495,456],{"class":222},[135,4497,481],{"class":222},[135,4499,723],{"class":145},[135,4501,481],{"class":222},[135,4503,693],{"class":222},[135,4505,4506,4508],{"class":137,"line":730},[135,4507,733],{"class":491},[135,4509,736],{"class":222},[135,4511,4512,4514,4516,4518,4520,4522],{"class":137,"line":739},[135,4513,742],{"class":222},[135,4515,745],{"class":145},[135,4517,481],{"class":222},[135,4519,750],{"class":222},[135,4521,753],{"class":600},[135,4523,693],{"class":222},[135,4525,4526,4528,4530,4532,4534,4536],{"class":137,"line":758},[135,4527,742],{"class":222},[135,4529,763],{"class":145},[135,4531,481],{"class":222},[135,4533,750],{"class":222},[135,4535,770],{"class":600},[135,4537,693],{"class":222},[135,4539,4540],{"class":137,"line":775},[135,4541,778],{"class":222},[135,4543,4544],{"class":137,"line":781},[135,4545,631],{"class":222},[16,4547,786],{},[125,4549,4550],{"className":189,"code":789,"filename":191,"language":192,"meta":130,"style":130},[132,4551,4552,4562,4572,4582,4592,4602],{"__ignoreMap":130},[135,4553,4554,4556,4558,4560],{"class":137,"line":138},[135,4555,283],{"class":203},[135,4557,456],{"class":222},[135,4559,800],{"class":411},[135,4561,590],{"class":222},[135,4563,4564,4566,4568,4570],{"class":137,"line":213},[135,4565,807],{"class":491},[135,4567,456],{"class":222},[135,4569,812],{"class":411},[135,4571,693],{"class":222},[135,4573,4574,4576,4578,4580],{"class":137,"line":243},[135,4575,819],{"class":491},[135,4577,456],{"class":222},[135,4579,824],{"class":411},[135,4581,693],{"class":222},[135,4583,4584,4586,4588,4590],{"class":137,"line":265},[135,4585,831],{"class":491},[135,4587,456],{"class":222},[135,4589,836],{"class":411},[135,4591,693],{"class":222},[135,4593,4594,4596,4598,4600],{"class":137,"line":291},[135,4595,843],{"class":491},[135,4597,456],{"class":222},[135,4599,848],{"class":411},[135,4601,693],{"class":222},[135,4603,4604],{"class":137,"line":316},[135,4605,631],{"class":222},[16,4607,857,4608,30],{},[26,4609,861],{"href":860},[40,4611,865],{"id":864},[16,4613,868],{},[16,4615,871,4616,30],{},[26,4617,875],{"href":874},[125,4619,4620],{"className":189,"code":878,"language":192,"meta":130,"style":130},[132,4621,4622],{"__ignoreMap":130},[135,4623,4624,4626,4628,4630,4632,4634,4636,4638,4640,4642,4644,4646,4648,4650,4652,4654,4656,4658,4660,4662],{"class":137,"line":138},[135,4625,885],{"class":203},[135,4627,456],{"class":222},[135,4629,459],{"class":203},[135,4631,30],{"class":222},[135,4633,465],{"class":464},[135,4635,30],{"class":222},[135,4637,898],{"class":464},[135,4639,30],{"class":222},[135,4641,903],{"class":411},[135,4643,478],{"class":222},[135,4645,908],{"class":491},[135,4647,456],{"class":222},[135,4649,481],{"class":222},[135,4651,915],{"class":145},[135,4653,481],{"class":222},[135,4655,488],{"class":222},[135,4657,922],{"class":491},[135,4659,456],{"class":222},[135,4661,927],{"class":600},[135,4663,631],{"class":222},[16,4665,932,4666,937,4668,30],{},[26,4667,936],{"href":935},[132,4669,940],{},[125,4671,4672],{"className":189,"code":943,"language":192,"meta":130,"style":130},[132,4673,4674,4690],{"__ignoreMap":130},[135,4675,4676,4678,4680,4682,4684,4686,4688],{"class":137,"line":138},[135,4677,951],{"class":950},[135,4679,954],{"class":141},[135,4681,478],{"class":222},[135,4683,959],{"class":141},[135,4685,30],{"class":222},[135,4687,964],{"class":141},[135,4689,967],{"class":222},[135,4691,4692,4694,4696],{"class":137,"line":213},[135,4693,972],{"class":203},[135,4695,750],{"class":222},[135,4697,977],{"class":141},[16,4699,980,4700,30],{},[26,4701,984],{"href":983},[125,4703,4704],{"className":189,"code":987,"filename":191,"language":192,"meta":130,"style":130},[132,4705,4706,4732,4742,4752,4762,4768],{"__ignoreMap":130},[135,4707,4708,4710,4712,4714,4716,4718,4720,4722,4724,4726,4728,4730],{"class":137,"line":138},[135,4709,994],{"class":203},[135,4711,488],{"class":222},[135,4713,999],{"class":203},[135,4715,456],{"class":222},[135,4717,459],{"class":203},[135,4719,30],{"class":222},[135,4721,465],{"class":464},[135,4723,30],{"class":222},[135,4725,898],{"class":464},[135,4727,30],{"class":222},[135,4729,1016],{"class":411},[135,4731,590],{"class":222},[135,4733,4734,4736,4738,4740],{"class":137,"line":213},[135,4735,1023],{"class":491},[135,4737,456],{"class":222},[135,4739,1028],{"class":411},[135,4741,693],{"class":222},[135,4743,4744,4746,4748,4750],{"class":137,"line":243},[135,4745,1035],{"class":491},[135,4747,456],{"class":222},[135,4749,1040],{"class":411},[135,4751,693],{"class":222},[135,4753,4754,4756,4758,4760],{"class":137,"line":265},[135,4755,1047],{"class":491},[135,4757,456],{"class":222},[135,4759,1052],{"class":600},[135,4761,693],{"class":222},[135,4763,4764,4766],{"class":137,"line":291},[135,4765,1059],{"class":491},[135,4767,1062],{"class":222},[135,4769,4770],{"class":137,"line":316},[135,4771,631],{"class":222},[16,4773,1069],{},[16,4775,1072],{},[48,4777,4778,4784,4790,4796],{},[51,4779,4780,1082],{},[57,4781,4782],{},[132,4783,1081],{},[51,4785,4786,1090],{},[57,4787,4788],{},[132,4789,1089],{},[51,4791,4792,1098],{},[57,4793,4794],{},[132,4795,1097],{},[51,4797,4798,1106],{},[57,4799,4800],{},[132,4801,1105],{},[16,4803,1109],{},[125,4805,4806],{"className":189,"code":1112,"filename":191,"language":192,"meta":130,"style":130},[132,4807,4808,4822,4840,4850,4858,4866],{"__ignoreMap":130},[135,4809,4810,4812,4814,4816,4818,4820],{"class":137,"line":138},[135,4811,1119],{"class":203},[135,4813,456],{"class":222},[135,4815,1124],{"class":203},[135,4817,30],{"class":222},[135,4819,1129],{"class":411},[135,4821,590],{"class":222},[135,4823,4824,4826,4828,4830,4832,4834,4836,4838],{"class":137,"line":213},[135,4825,1136],{"class":491},[135,4827,1139],{"class":222},[135,4829,459],{"class":411},[135,4831,30],{"class":222},[135,4833,1146],{"class":464},[135,4835,30],{"class":222},[135,4837,940],{"class":464},[135,4839,693],{"class":222},[135,4841,4842,4844,4846,4848],{"class":137,"line":243},[135,4843,1157],{"class":491},[135,4845,1139],{"class":222},[135,4847,1162],{"class":600},[135,4849,693],{"class":222},[135,4851,4852,4854,4856],{"class":137,"line":265},[135,4853,1169],{"class":491},[135,4855,1139],{"class":222},[135,4857,1174],{"class":222},[135,4859,4860,4862,4864],{"class":137,"line":291},[135,4861,1179],{"class":491},[135,4863,1139],{"class":222},[135,4865,1174],{"class":222},[135,4867,4868],{"class":137,"line":316},[135,4869,631],{"class":222},[16,4871,1190,4872,1194,4874,1199,4876,1202,4878,30],{},[132,4873,1193],{},[1196,4875,1198],{},[132,4877,1089],{},[132,4879,1040],{},[40,4881,1208],{"id":1207},[16,4883,1211],{},[125,4885,4886],{"className":189,"code":1214,"filename":191,"language":192,"meta":130,"style":130},[132,4887,4888],{"__ignoreMap":130},[135,4889,4890,4892,4894,4896,4898,4900,4902,4904],{"class":137,"line":138},[135,4891,1221],{"class":203},[135,4893,456],{"class":222},[135,4895,1226],{"class":203},[135,4897,30],{"class":222},[135,4899,533],{"class":411},[135,4901,478],{"class":222},[135,4903,994],{"class":411},[135,4905,631],{"class":222},[16,4907,1239,4908,1243],{},[132,4909,1242],{},[16,4911,1246,4912,1250],{},[132,4913,1249],{},[125,4915,4916],{"className":189,"code":1253,"filename":191,"language":192,"meta":130,"style":130},[132,4917,4918],{"__ignoreMap":130},[135,4919,4920,4922,4924,4926,4928,4930,4932,4934,4936,4938,4940,4942,4944,4946],{"class":137,"line":138},[135,4921,1221],{"class":203},[135,4923,456],{"class":222},[135,4925,1264],{"class":203},[135,4927,30],{"class":222},[135,4929,1129],{"class":411},[135,4931,478],{"class":222},[135,4933,1249],{"class":491},[135,4935,456],{"class":222},[135,4937,959],{"class":411},[135,4939,30],{"class":222},[135,4941,1146],{"class":464},[135,4943,30],{"class":222},[135,4945,1242],{"class":464},[135,4947,631],{"class":222},[16,4949,1289],{},[125,4951,4952],{"className":189,"code":1292,"filename":191,"language":192,"meta":130,"style":130},[132,4953,4954],{"__ignoreMap":130},[135,4955,4956,4958,4960,4962,4964],{"class":137,"line":138},[135,4957,1299],{"class":203},[135,4959,456],{"class":222},[135,4961,1304],{"class":203},[135,4963,1307],{"class":222},[135,4965,1310],{"class":203},[40,4967,103],{"id":1313},[16,4969,1316],{},[176,4971,1320],{"id":1319},[16,4973,1323],{},[16,4975,1326],{},[16,4977,1329,4978,1332],{},[132,4979,172],{},[125,4981,4982],{"className":189,"code":1335,"filename":191,"language":192,"meta":130,"style":130},[132,4983,4984,4996,5004,5016,5042],{"__ignoreMap":130},[135,4985,4986,4988,4990,4992,4994],{"class":137,"line":138},[135,4987,1342],{"class":950},[135,4989,1345],{"class":411},[135,4991,478],{"class":222},[135,4993,812],{"class":491},[135,4995,967],{"class":222},[135,4997,4998,5000,5002],{"class":137,"line":213},[135,4999,1356],{"class":203},[135,5001,456],{"class":222},[135,5003,1361],{"class":222},[135,5005,5006,5008,5010,5012,5014],{"class":137,"line":243},[135,5007,1366],{"class":199},[135,5009,1369],{"class":203},[135,5011,1372],{"class":199},[135,5013,1375],{"class":203},[135,5015,1378],{"class":222},[135,5017,5018,5020,5022,5024,5026,5028,5030,5032,5034,5036,5038,5040],{"class":137,"line":265},[135,5019,1383],{"class":203},[135,5021,30],{"class":222},[135,5023,1388],{"class":411},[135,5025,478],{"class":222},[135,5027,1393],{"class":141},[135,5029,478],{"class":222},[135,5031,1398],{"class":411},[135,5033,658],{"class":222},[135,5035,481],{"class":222},[135,5037,172],{"class":145},[135,5039,481],{"class":222},[135,5041,1409],{"class":222},[135,5043,5044,5046,5048,5050,5052,5054,5056,5058],{"class":137,"line":291},[135,5045,1414],{"class":199},[135,5047,1417],{"class":222},[135,5049,1417],{"class":222},[135,5051,30],{"class":222},[135,5053,1424],{"class":411},[135,5055,478],{"class":222},[135,5057,1429],{"class":411},[135,5059,631],{"class":222},[1433,5061,5062],{},[16,5063,5064,1440,5066,30],{},[57,5065,1439],{},[132,5067,1089],{},[16,5069,1445,5070,1450],{},[26,5071,1449],{"href":1448},[125,5073,5074],{"className":189,"code":1453,"filename":191,"language":192,"meta":130,"style":130},[132,5075,5076,5086,5108,5122,5130,5156,5160,5166,5170,5174,5192,5226],{"__ignoreMap":130},[135,5077,5078,5080,5082,5084],{"class":137,"line":138},[135,5079,1460],{"class":222},[135,5081,959],{"class":411},[135,5083,30],{"class":222},[135,5085,1467],{"class":411},[135,5087,5088,5090,5092,5094,5096,5098,5100,5102,5104,5106],{"class":137,"line":213},[135,5089,1342],{"class":950},[135,5091,1474],{"class":411},[135,5093,478],{"class":222},[135,5095,812],{"class":491},[135,5097,488],{"class":222},[135,5099,1483],{"class":491},[135,5101,1486],{"class":222},[135,5103,1489],{"class":222},[135,5105,1492],{"class":141},[135,5107,1378],{"class":222},[135,5109,5110,5112,5114,5116,5118,5120],{"class":137,"line":243},[135,5111,1499],{"class":203},[135,5113,456],{"class":222},[135,5115,1345],{"class":411},[135,5117,478],{"class":222},[135,5119,812],{"class":411},[135,5121,631],{"class":222},[135,5123,5124,5126,5128],{"class":137,"line":265},[135,5125,1514],{"class":203},[135,5127,456],{"class":222},[135,5129,1519],{"class":222},[135,5131,5132,5134,5136,5138,5140,5142,5144,5146,5148,5150,5152,5154],{"class":137,"line":291},[135,5133,1524],{"class":950},[135,5135,1527],{"class":145},[135,5137,1530],{"class":203},[135,5139,1533],{"class":600},[135,5141,1536],{"class":203},[135,5143,1539],{"class":600},[135,5145,1542],{"class":203},[135,5147,1545],{"class":145},[135,5149,1548],{"class":600},[135,5151,1081],{"class":203},[135,5153,1539],{"class":600},[135,5155,626],{"class":145},[135,5157,5158],{"class":137,"line":316},[135,5159,1559],{"class":222},[135,5161,5162,5164],{"class":137,"line":341},[135,5163,1414],{"class":199},[135,5165,1566],{"class":203},[135,5167,5168],{"class":137,"line":700},[135,5169,406],{"emptyLinePlaceholder":405},[135,5171,5172],{"class":137,"line":713},[135,5173,406],{"emptyLinePlaceholder":405},[135,5175,5176,5178,5180,5182,5184,5186,5188,5190],{"class":137,"line":730},[135,5177,1579],{"class":203},[135,5179,456],{"class":222},[135,5181,1584],{"class":203},[135,5183,1307],{"class":222},[135,5185,1589],{"class":203},[135,5187,30],{"class":222},[135,5189,1129],{"class":411},[135,5191,590],{"class":222},[135,5193,5194,5196,5198,5200,5202,5204,5206,5208,5210,5212,5214,5216,5218,5220,5222,5224],{"class":137,"line":739},[135,5195,1600],{"class":491},[135,5197,456],{"class":222},[135,5199,1605],{"class":411},[135,5201,478],{"class":222},[135,5203,959],{"class":411},[135,5205,30],{"class":222},[135,5207,1146],{"class":464},[135,5209,30],{"class":222},[135,5211,1249],{"class":464},[135,5213,488],{"class":222},[135,5215,459],{"class":411},[135,5217,30],{"class":222},[135,5219,1146],{"class":464},[135,5221,30],{"class":222},[135,5223,1081],{"class":464},[135,5225,631],{"class":222},[135,5227,5228],{"class":137,"line":758},[135,5229,631],{"class":222},[176,5231,1639],{"id":1638},[16,5233,1642,5234,1647,5236,1654],{},[26,5235,1646],{"href":1645},[26,5237,5238],{"href":1650},[132,5239,1653],{},[125,5241,5242],{"className":189,"code":1657,"filename":191,"language":192,"meta":130,"style":130},[132,5243,5244,5258,5272,5296],{"__ignoreMap":130},[135,5245,5246,5248,5250,5252,5254,5256],{"class":137,"line":138},[135,5247,1664],{"class":203},[135,5249,456],{"class":222},[135,5251,1669],{"class":203},[135,5253,30],{"class":222},[135,5255,1653],{"class":411},[135,5257,590],{"class":222},[135,5259,5260,5262,5264,5266,5268,5270],{"class":137,"line":213},[135,5261,1680],{"class":491},[135,5263,456],{"class":222},[135,5265,481],{"class":222},[135,5267,1687],{"class":145},[135,5269,481],{"class":222},[135,5271,693],{"class":222},[135,5273,5274,5276,5278,5280,5282,5284,5286,5288,5290,5292,5294],{"class":137,"line":243},[135,5275,1696],{"class":491},[135,5277,456],{"class":222},[135,5279,369],{"class":411},[135,5281,30],{"class":222},[135,5283,655],{"class":464},[135,5285,658],{"class":222},[135,5287,481],{"class":222},[135,5289,663],{"class":145},[135,5291,481],{"class":222},[135,5293,1715],{"class":222},[135,5295,1719],{"class":1718},[135,5297,5298],{"class":137,"line":265},[135,5299,631],{"class":222},[16,5301,1726,5302,1730],{},[132,5303,1729],{},[176,5305,1734],{"id":1733},[16,5307,1737,5308,1740,5310,1747,5314,30],{},[132,5309,1653],{},[26,5311,5312],{"href":1743},[132,5313,1746],{},[132,5315,1729],{},[125,5317,5318],{"className":189,"code":1752,"filename":191,"language":192,"meta":130,"style":130},[132,5319,5320,5334,5386,5396,5418,5422],{"__ignoreMap":130},[135,5321,5322,5324,5326,5328,5330,5332],{"class":137,"line":138},[135,5323,1759],{"class":203},[135,5325,456],{"class":222},[135,5327,1764],{"class":203},[135,5329,30],{"class":222},[135,5331,1129],{"class":411},[135,5333,590],{"class":222},[135,5335,5336,5338,5340,5342,5344,5346,5348,5350,5352,5354,5356,5358,5360,5362,5364,5366,5368,5370,5372,5374,5376,5378,5380,5382,5384],{"class":137,"line":213},[135,5337,1775],{"class":222},[135,5339,959],{"class":411},[135,5341,30],{"class":222},[135,5343,1146],{"class":464},[135,5345,30],{"class":222},[135,5347,1786],{"class":411},[135,5349,478],{"class":222},[135,5351,959],{"class":411},[135,5353,30],{"class":222},[135,5355,1146],{"class":464},[135,5357,30],{"class":222},[135,5359,1081],{"class":464},[135,5361,488],{"class":222},[135,5363,459],{"class":411},[135,5365,30],{"class":222},[135,5367,1146],{"class":464},[135,5369,30],{"class":222},[135,5371,1811],{"class":464},[135,5373,488],{"class":222},[135,5375,459],{"class":411},[135,5377,30],{"class":222},[135,5379,1146],{"class":464},[135,5381,30],{"class":222},[135,5383,1249],{"class":464},[135,5385,1826],{"class":222},[135,5387,5388,5390,5392,5394],{"class":137,"line":243},[135,5389,1831],{"class":491},[135,5391,456],{"class":222},[135,5393,1729],{"class":411},[135,5395,590],{"class":222},[135,5397,5398,5400,5402,5404,5406,5408,5410,5412,5414,5416],{"class":137,"line":265},[135,5399,1842],{"class":411},[135,5401,30],{"class":222},[135,5403,1746],{"class":411},[135,5405,478],{"class":222},[135,5407,959],{"class":411},[135,5409,30],{"class":222},[135,5411,1146],{"class":464},[135,5413,30],{"class":222},[135,5415,1811],{"class":464},[135,5417,1826],{"class":222},[135,5419,5420],{"class":137,"line":291},[135,5421,1865],{"class":222},[135,5423,5424],{"class":137,"line":316},[135,5425,631],{"class":222},[40,5427,1873],{"id":1872},[16,5429,1876],{},[125,5431,5432],{"className":189,"code":1879,"filename":191,"language":192,"meta":130,"style":130},[132,5433,5434],{"__ignoreMap":130},[135,5435,5436,5438,5440,5442],{"class":137,"line":138},[135,5437,1886],{"class":411},[135,5439,478],{"class":222},[135,5441,1891],{"class":411},[135,5443,631],{"class":222},[40,5445,1897],{"id":1896},[16,5447,1900],{},[16,5449,1903,5450],{},[132,5451,1906],{},[125,5453,5454],{"className":189,"code":1909,"filename":191,"language":192,"meta":130,"style":130},[132,5455,5456],{"__ignoreMap":130},[135,5457,5458,5460,5462,5464],{"class":137,"line":138},[135,5459,959],{"class":203},[135,5461,30],{"class":222},[135,5463,1920],{"class":411},[135,5465,415],{"class":222},[16,5467,1925],{},[125,5469,5470],{"className":127,"code":1928,"language":129,"meta":130,"style":130},[132,5471,5472],{"__ignoreMap":130},[135,5473,5474,5476],{"class":137,"line":138},[135,5475,192],{"class":141},[135,5477,1937],{"class":145},[16,5479,1940],{},[125,5481,5482],{"className":127,"code":1943,"language":129,"meta":130,"style":130},[132,5483,5484],{"__ignoreMap":130},[135,5485,5486,5488,5490,5492,5494,5496],{"class":137,"line":138},[135,5487,1950],{"class":141},[135,5489,1953],{"class":145},[135,5491,1956],{"class":222},[135,5493,1959],{"class":145},[135,5495,1962],{"class":222},[135,5497,1965],{"class":145},[16,5499,1968],{},[40,5501,1972],{"id":1971},[16,5503,1975],{},[16,5505,1978],{},[16,5507,1981],{},[40,5509,1985],{"id":1984},[16,5511,1988],{},[16,5513,1991,5514,1996,5516,2001,5518,2006,5520,30],{},[26,5515,1995],{"href":1994},[26,5517,2000],{"href":1999},[26,5519,2005],{"href":2004},[26,5521,2010],{"href":2009},[16,5523,2013],{},[2015,5525,2017],{},{"title":130,"searchDepth":213,"depth":213,"links":5527},[5528,5529,5532,5533,5534,5535,5540,5541,5542,5543],{"id":42,"depth":213,"text":43},{"id":119,"depth":213,"text":120,"children":5530},[5531],{"id":178,"depth":243,"text":179},{"id":418,"depth":213,"text":59},{"id":864,"depth":213,"text":865},{"id":1207,"depth":213,"text":1208},{"id":1313,"depth":213,"text":103,"children":5536},[5537,5538,5539],{"id":1319,"depth":243,"text":1320},{"id":1638,"depth":243,"text":1639},{"id":1733,"depth":243,"text":1734},{"id":1872,"depth":213,"text":1873},{"id":1896,"depth":213,"text":1897},{"id":1971,"depth":213,"text":1972},{"id":1984,"depth":213,"text":1985},{"date":2039,"thumbnail":130,"tags":5545,"keywords":5546},[2041,2042,2043],[2042,2045,2046,230],{"title":5,"description":2036},{"loc":2047},[5550,5551],{"title":2588,"path":2589,"stem":2590,"children":-1},{"title":2601,"path":2602,"stem":2603,"children":-1},1781214763102]