for developers

pw.xpacks.llm.servers

class pw.xpacks.llm.servers.DocumentStoreServer(host, port, document_store, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of DocumentStore. It exposes three endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/inputs which is answered using list_documents method,
  • Parameters
    • host (-) – host on which server will run
    • port (-) – port on which server will run
    • document_store (-) – instance of DocumentStore which is used to answer queries received in the endpoints.
    • rest_kwargs (-) – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

  • Parameters
    • threaded (bool) – if True, the server will be run in a new thread.
    • with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
    • cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
    • **kwargs – optional kwargs to be passed to pw.run.

class pw.xpacks.llm.servers.QARestServer(host, port, rag_question_answerer, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of BaseQuestionAnswerer. It exposes four endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/pw_list_documents which is answered using list_documents method,
  • /v1/pw_ai_answer which is answered using answer_query method,
  • Parameters
    • host (-) – host on which server will run
    • port (-) – port on which server will run
    • rag_question_answerer (-) – instance of BaseQuestionAnswerer which is used to answer queries received in the endpoints.
    • rest_kwargs (-) – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

  • Parameters
    • threaded (bool) – if True, the server will be run in a new thread.
    • with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
    • cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
    • **kwargs – optional kwargs to be passed to pw.run.

class pw.xpacks.llm.servers.QASummaryRestServer(host, port, rag_question_answerer, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of SummaryQuestionAnswerer. It exposes five endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/pw_list_documents which is answered using list_documents method,
  • /v1/pw_ai_answer which is answered using answer method,
  • /v1/pw_ai_summary which is answered using summarize_query method.
  • Parameters
    • host (-) – host on which server will run
    • port (-) – port on which server will run
    • rag_question_answerer (-) – instance of SummaryQuestionAnswerer which is used to answer queries received in the endpoints.
    • rest_kwargs (-) – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), **kwargs)

sourceStart the server. Enabling persistence will cache the UDFs for which cache_strategy is set.

  • Parameters
    • threaded (bool) – if True, the server will be run in a new thread.
    • with_cache (bool) – if True, caching will be enabled for the UDFs for which cache_strategy is set.
    • cache_backend (Backend | None) – backend used for caching. Only relevant if with_cache is set to True.
    • **kwargs – optional kwargs to be passed to pw.run.