pw.debug

Methods and classes for debugging Pathway computation.

Typical use:

import pathway as pw
t1 = pw.debug.table_from_markdown('''
pet
Dog
Cat
''')
t2 = t1.select(animal=t1.pet, desc="fluffy")
pw.debug.compute_and_print(t2, include_id=False)

animal | desc
Cat    | fluffy
Dog    | fluffy

compute_and_print(*tables, include_id=True, short_pointers=True, n_rows=None, **kwargs)

sourceA function running the computations and printing the table.

• Parameters
  • tables (Table) – tables to be computed and printed
  • include_id – whether to show ids of rows
  • short_pointers – whether to shorten printed ids
  • n_rows (int | None) – number of rows to print, if None whole table will be printed

compute_and_print_update_stream(*tables, include_id=True, short_pointers=True, n_rows=None, **kwargs)

sourceA function running the computations and printing the update stream of the table.

• Parameters
  • tables (Table) – tables for which the update stream is to be computed and printed
  • include_id – whether to show ids of rows
  • short_pointers – whether to shorten printed ids
  • n_rows (int | None) – number of rows to print, if None whole update stream will be printed

table_from_markdown(table_def, id_from=None, unsafe_trusted_ids=False, schema=None, *, split_on_whitespace=True, )

sourceA function for creating a table from its definition in markdown. If it contains a special column __time__, rows will be split into batches with timestamps from the column. A special column __diff__ can be used to set an event type - with 1 treated as inserting the row and -1 as removing it.

By default, it splits on whitespaces. To get a table containing strings with whitespaces, use with split_on_whitespace = False.

table_from_pandas(df, id_from=None, unsafe_trusted_ids=False, schema=None, )

sourceA function for creating a table from a pandas DataFrame. If it contains a special column __time__, rows will be split into batches with timestamps from the column. A special column __diff__ can be used to set an event type - with 1 treated as inserting the row and -1 as removing it.

table_from_parquet(path, id_from=None, unsafe_trusted_ids=False, )

sourceReads a Parquet file into a pandas DataFrame and then converts that into a Pathway table.

table_from_rows(schema, rows, unsafe_trusted_ids=False, is_stream=False)

sourceA function for creating a table from a list of tuples. Each tuple should describe one row of the input data (or stream), matching provided schema.

If is_stream is set to True, each tuple representing a row should contain two additional columns, the first indicating the time of arrival of particular row and the second indicating whether the row should be inserted (1) or deleted (-1).

table_to_dicts(table, **kwargs)

sourceRuns the computations needed to get the contents of the Pathway Table and converts it to a dictionary representation, where each column is mapped to its respective values.

• Parameters
  • table (Table) – The Pathway Table to be converted.
  • **kwargs – Additional keyword arguments to customize the behavior of the function.
    • terminate_on_error (bool): If True, the function will terminate execution upon encountering an error during the squashing of updates. Defaults to True.
• Returns
  tuple – A tuple containing two elements: 1) list of keys (pointers) that represent the rows in the table, and 2) a dictionary where each key is a column name, and the value is another dictionary mapping row keys (pointers) to their respective values in that column.

table_to_parquet(table, filename)

sourceConverts a Pathway Table into a pandas DataFrame and then writes it to Parquet