TEST

pw.xpacks.llm.servers

class pw.xpacks.llm.servers.DocumentStoreServer(host, port, document_store, **rest_kwargs)

Creates a REST Server for answering queries to a given instance of DocumentStore. It exposes three endpoints:

/v1/retrieve which is answered using retrieve method,
/v1/statistics which is answered using statistics method,
/v1/inputs which is answered using list_documents method,

Parameters
- host (str) – host on which server will run
- port (int) – port on which server will run
- document_store (DocumentStore) – instance of DocumentStore which is used to answer queries received in the endpoints.
- rest_kwargs – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

Parameters
- threaded (bool) – if True, the server will be run in a new thread.
- with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
- cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
- **kwargs – optional kwargs to be passed to pw.run.

class pw.xpacks.llm.servers.QARestServer(host, port, rag_question_answerer, **rest_kwargs)

Creates a REST Server for answering queries to a given instance of BaseQuestionAnswerer. It exposes four endpoints:

/v1/retrieve which is answered using retrieve method,
/v1/statistics which is answered using statistics method,
/v1/pw_list_documents which is answered using list_documents method,
/v1/pw_ai_answer which is answered using answer_query method,

Parameters
- host (str) – host on which server will run
- port (int) – port on which server will run
- rag_question_answerer (BaseQuestionAnswerer) – instance of BaseQuestionAnswerer which is used to answer queries received in the endpoints.
- rest_kwargs – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

Parameters
- threaded (bool) – if True, the server will be run in a new thread.
- with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
- cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
- **kwargs – optional kwargs to be passed to pw.run.

class pw.xpacks.llm.servers.QASummaryRestServer(host, port, rag_question_answerer, **rest_kwargs)

Creates a REST Server for answering queries to a given instance of SummaryQuestionAnswerer. It exposes five endpoints:

/v1/retrieve which is answered using retrieve method,
/v1/statistics which is answered using statistics method,
/v1/pw_list_documents which is answered using list_documents method,
/v1/pw_ai_answer which is answered using answer method,
/v1/pw_ai_summary which is answered using summarize_query method.

Parameters
- host (str) – host on which server will run
- port (int) – port on which server will run
- rag_question_answerer (SummaryQuestionAnswerer) – instance of SummaryQuestionAnswerer which is used to answer queries received in the endpoints.
- rest_kwargs – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

Parameters
- threaded (bool) – if True, the server will be run in a new thread.
- with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
- cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
- **kwargs – optional kwargs to be passed to pw.run.