for developers

pw.xpacks.llm.servers

class pw.xpacks.llm.servers.DocumentStoreServer(host, port, document_store, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of DocumentStore. It exposes three endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/inputs which is answered using list_documents method,
  • Parameters
    • host (-) – host on which server will run
    • port (-) – port on which server will run
    • document_store (-) – instance of DocumentStore which is used to answer queries received in the endpoints.
    • rest_kwargs (-) – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), *args, **kwargs)

sourceStart the app with cache configs. Enabling persistence will cache the embedding, and LLM requests between the runs.

class pw.xpacks.llm.servers.QARestServer(host, port, rag_question_answerer, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of BaseQuestionAnswerer. It exposes four endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/pw_list_documents which is answered using list_documents method,
  • /v1/pw_ai_answer which is answered using answer_query method,
  • Parameters
    • host (-) – host on which server will run
    • port (-) – port on which server will run
    • rag_question_answerer (-) – instance of BaseQuestionAnswerer which is used to answer queries received in the endpoints.
    • rest_kwargs (-) – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), *args, **kwargs)

sourceStart the app with cache configs. Enabling persistence will cache the embedding, and LLM requests between the runs.

class pw.xpacks.llm.servers.QASummaryRestServer(host, port, rag_question_answerer, **rest_kwargs)

[source]

Creates a REST Server for answering queries to a given instance of SummaryQuestionAnswerer. It exposes five endpoints:

  • /v1/retrieve which is answered using retrieve method,
  • /v1/statistics which is answered using statistics method,
  • /v1/pw_list_documents which is answered using list_documents method,
  • /v1/pw_ai_answer which is answered using answer method,
  • /v1/pw_ai_summary which is answered using summarize_query method.
  • Parameters
    • host (-) – host on which server will run
    • port (-) – port on which server will run
    • rag_question_answerer (-) – instance of SummaryQuestionAnswerer which is used to answer queries received in the endpoints.
    • rest_kwargs (-) – optional kwargs to be passed to pw.io.http.rest_connector

run(threaded=False, with_cache=True, cache_backend=pw.persistence.Backend.filesystem('./Cache'), *args, **kwargs)

sourceStart the app with cache configs. Enabling persistence will cache the embedding, and LLM requests between the runs.