for developers
20k

pw.io.elasticsearch

write(table, host, auth, index_name, *, name=None, sort_by=None)

sourceWrite a table to a given index in ElasticSearch.

  • Parameters
    • table (Table) – the table to output.
    • host (str) – the host and port, on which Elasticsearch server works.
    • auth (ElasticSearchAuth) – credentials for Elasticsearch authorization.
    • index_name (str) – name of the index, which gets the docs.
    • name (str | None) – A unique name for the connector. If provided, this name will be used in logs and monitoring dashboards.
    • sort_by (Optional[Iterable[ColumnReference]]) – If specified, the output will be sorted in ascending order based on the values of the given columns within each minibatch. When multiple columns are provided, the corresponding value tuples will be compared lexicographically.
  • Returns
    None

Example:

Consider there is an instance of Elasticsearch, running locally on a port 9200. There we have an index “animals”, containing an information about pets and their owners.

For the sake of simplicity we will also consider that the cluster has a simple username-password authentication having both username and password equal to “admin”.

Now suppose we want to send a Pathway table pets to this local instance of Elasticsearch.

import pathway as pw
pets = pw.debug.table_from_markdown('''
age | owner | pet
10  | Alice | dog
9   | Bob   | cat
8   | Alice | cat
''')

It can be done as follows:

pw.io.elasticsearch.write(
    table=pets,
    host="http://localhost:9200",
    auth=pw.io.elasticsearch.ElasticSearchAuth.basic("admin", "admin"),
    index_name="animals",
)

All the updates of table “pets” will be indexed to “animals” as well.