List of updates for version 2022.11

[root-11]

• <-- tickmark means done in branch v2022_11_0. See comments for commit references.

Config

• Setting so that tablite can run single core only if educators wants this.
• allow multiple h5 files. tempdir is thereby the default storage, but user can point individual table to other storages. Make it possible to work with multiple HDF repos #22
• allow user to set h5 file to "memory" and exclusively use RAM.

new tools.py

• date_range(start, stop, step) like 2022/1/1, 2023/1/1, timedelta(days1) # returns list of datetimes.
• add datatypes.guess
• add xround

new Table methods

• table.remove_duplicate_rows()
• table.drop_na(*arg) removes rows with and None, np.nan
• table.replace(target, replacement) which searches across all columns, e.g. t.replace(None, -1)
• table.replace_missing_values(source=[...], target=column_name) which looks up nearest neighbour in sources and substitute into target. implement replace missing values with MP support #18
• table.to_pandas()
• Table.from_pandas(pd.DataFrame)
• table.to_h5()
• Table.from_h5()
• table.to_dict(columns, slice) # to_dict returns python dict with column names as keys and lists of values as values. The optional slice permits an effective retrieval of a subset of rows.
• Table.from_dict()
• table.to_list() returns list of column names, + list of each column
• Table.from_list(column names + data) (the inverse to .to_list()
• table.transpose(columns=['Monday', 'Tuesday','Wednesday', 'Thursday', 'Friday'], as='day') turns the columns into a single column under the heading of 'day'
• complete the list of importable formats.
• remove requirement for column name declaration

new Column methods:

• Column[:] returns list of native python types. No more numpy arrays or numpy types.
• Column.to_numpy(slice) returns numpy's ndarray.

Documentation

• feature matrix: comparison of pandas and Tablite
• allow highlights in html output in jupyter

Cleaner code:

• test for page size limit. Add config to break early (90% of limit) or break late (110% of limit).
• test for various np broadcast functions.

[root-11]

See commit cb337f2 for date_range, xround and guess

[root-11]

See commit 9c82080 for drop

[root-11]

Tablite 2022.11 is available in pre-release as 2022.11.dev1 on pypi.

[root-11]

Added completed changes in the changelog. https://github.com/root-11/tablite/blob/master/changelog.md

[root-11]

All completed in 2023.