pw.xpacks.llm.embedders

Model has to be specified either in constructor call or in each application, no default is provided. The capacity, retry_strategy and cache_strategy need to be specified during object construction. All other arguments can be overridden during application.

• Parameters
  • capacity (-) – Maximum number of concurrent operations allowed. Defaults to None, indicating no specific limit.
  • retry_strategy (-) – Strategy for handling retries in case of failures. Defaults to None, meaning no retries.
  • cache_strategy (-) – Defines the caching mechanism. To enable caching, a valid CacheStrategy should be provided. See Cache strategy for more information. Defaults to None.
  • model (-) – The embedding model to use.
  • timeout (-) – The timeout value for the API call, default 10 mins
  • litellm_call_id (-) – The call ID for litellm logging.
  • litellm_logging_obj (-) – The litellm logging object.
  • logger_fn (-) – The logger function.
  • api_base (-) – Optional. The base URL for the API.
  • api_version (-) – Optional. The version of the API.
  • api_key (-) – Optional. The API key to use.
  • api_type (-) – Optional. The type of the API.
  • custom_llm_provider (-) – The custom llm provider.

Any arguments can be provided either to the constructor or in the UDF call. To specify the model in the UDF call, set it to None.

Example:

import pathway as pw
from pathway.xpacks.llm import embedders
embedder = embedders.LiteLLMEmbedder(model="text-embedding-ada-002")
t = pw.debug.table_from_markdown('''
txt
Text
''')
t.select(ret=embedder(pw.this.txt))

<pathway.Table schema={'ret': list[float]}>

import pathway as pw
from pathway.xpacks.llm import embedders
embedder = embedders.LiteLLMEmbedder()
t = pw.debug.table_from_markdown('''
txt  | model
Text | text-embedding-ada-002
''')
t.select(ret=embedder(pw.this.txt, model=pw.this.model))

<pathway.Table schema={'ret': list[float]}>

The capacity, retry_strategy and cache_strategy need to be specified during object construction. All other arguments can be overridden during application.

• Parameters
  • capacity (-) – Maximum number of concurrent operations allowed. Defaults to None, indicating no specific limit.
  • retry_strategy (-) – Strategy for handling retries in case of failures. Defaults to None, meaning no retries.
  • cache_strategy (-) – Defines the caching mechanism. To enable caching, a valid CacheStrategy should be provided. See Cache strategy for more information. Defaults to None.
  • model (-) – ID of the model to use. You can use the List models API to see all of your available models, or see Model overview for descriptions of them.
  • encoding_format (-) – The format to return the embeddings in. Can be either float or base64.
  • user (-) – A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. Learn more.
  • extra_headers (-) – Send extra headers
  • extra_query (-) – Add additional query parameters to the request
  • extra_body (-) – Add additional JSON properties to the request
  • timeout (-) – Timeout for requests, in seconds

Any arguments can be provided either to the constructor or in the UDF call. To specify the model in the UDF call, set it to None.

Example:

import pathway as pw
from pathway.xpacks.llm import embedders
embedder = embedders.OpenAIEmbedder(model="text-embedding-ada-002")
t = pw.debug.table_from_markdown('''
txt
Text
''')
t.select(ret=embedder(pw.this.txt))

<pathway.Table schema={'ret': list[float]}>

import pathway as pw
from pathway.xpacks.llm import embedders
embedder = embedders.OpenAIEmbedder()
t = pw.debug.table_from_markdown('''
txt  | model
Text | text-embedding-ada-002
''')
t.select(ret=embedder(pw.this.txt, model=pw.this.model))

<pathway.Table schema={'ret': list[float]}>

• Parameters
  • model (str) – model name or path
  • call_kwargs (dict) – kwargs that will be passed to each call of encode. These can be overridden during each application. For possible arguments check the Sentence-Transformers documentation.
  • device (str) – defines which device will be used to run the Pipeline
  • sentencetransformer_kwargs – kwargs accepted during initialization of SentenceTransformers. For possible arguments check the Sentence-Transformers documentation

Example:

import pathway as pw
from pathway.xpacks.llm import embedders
embedder = embedders.SentenceTransformerEmbedder(model="intfloat/e5-large-v2")
t = pw.debug.table_from_markdown('''
txt
Text
''')
t.select(ret=embedder(pw.this.txt))

<pathway.Table schema={'ret': list[float]}>