62k

pw.xpacks.llm.servers

class DocumentStoreServer(host, port, document_store, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of DocumentStore. It exposes three endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/inputs which is answered using list_documents method,
  • Parameters
    • host (str) – host on which server will run
    • port (int) – port on which server will run
    • document_store (DocumentStore) – instance of DocumentStore which is used to answer queries received in the endpoints.
    • rest_kwargs – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

  • Parameters
    • threaded (bool) – if True, the server will be run in a new thread.
    • with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
    • cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
    • **kwargs – optional kwargs to be passed to pw.run.

class QARestServer(host, port, rag_question_answerer, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of BaseQuestionAnswerer. It exposes four endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/pw_list_documents which is answered using list_documents method,
  • /v1/pw_ai_answer which is answered using answer_query method,
  • /v2/list_documents which is answered using list_documents method,
  • /v2/answer which is answered using answer_query method,
  • Parameters
    • host (str) – host on which server will run
    • port (int) – port on which server will run
    • rag_question_answerer (BaseQuestionAnswerer) – instance of BaseQuestionAnswerer which is used to answer queries received in the endpoints.
    • rest_kwargs – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

  • Parameters
    • threaded (bool) – if True, the server will be run in a new thread.
    • with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
    • cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
    • **kwargs – optional kwargs to be passed to pw.run.

class QASummaryRestServer(host, port, rag_question_answerer, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of SummaryQuestionAnswerer. It exposes five endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/pw_list_documents which is answered using list_documents method,
  • /v1/pw_ai_answer which is answered using answer method,
  • /v1/pw_ai_summary which is answered using summarize_query method.
  • /v2/list_documents which is answered using list_documents method,
  • /v2/answer which is answered using answer_query method,
  • /v2/summarize which is answered using summarize_query method.
  • Parameters
    • host (str) – host on which server will run
    • port (int) – port on which server will run
    • rag_question_answerer (SummaryQuestionAnswerer) – instance of SummaryQuestionAnswerer which is used to answer queries received in the endpoints.
    • rest_kwargs – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

  • Parameters
    • threaded (bool) – if True, the server will be run in a new thread.
    • with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
    • cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
    • **kwargs – optional kwargs to be passed to pw.run.