Skip to content
This repository was archived by the owner on Apr 2, 2026. It is now read-only.

2023.6.dev0 - Breaking release with ~6x speedup.

Pre-release
Pre-release

Choose a tag to compare

@root-11 root-11 released this 06 Jun 17:22

This release 2023.6.dev0 addresses #61 with a significant number of changes and features ~6x speedup.

For a quick summary view the tutorial

Changes

A new Config-class available in config.py, which sets:

  • config.page_size = 1_000_000
  • config.workdir = tempfile.gettempdir()

The workdir uses ///pages for managing pages, so that different python processes don't interact with the same storage location. Previously two python processes would enter conflict on the same tablite.h5-file.

Table now accepts the keyword columns as a dict in init:

     t = Table(columns={'b':[4,5,6], 'c':[7,8,9]})`

Table now accepts header/data combinations in init:

    t = Table(header=['b','c'], data=[[4,5,6],[7,8,9]])`

With these features it is no longer necessary to write:

    t = Table
    t['b'] = [4,5,6]
    t['c'] = [7,8,9]

Preferred approach for subclassing tables is:

class MyTable(tablite.Table):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.x = kwargs.get("x", 42)  # <== special variable required on MyTable.

    def copy(self):
        # tablite.Table implements
        # new = cls();   # self.x is now 42 !!!
        # for name,column in self.items():
        #     new[name] = column

        # MyTable therefore implements:
        cp = super(MyTable, self).copy()
        cp.x = self.x  # updating to real x.
        return cp

Replace is refactored from using a single value to a mapping,

    example:
    >>> t = Table(columns={'A': [1,2,3,4]})
    >>> t['A'].replace({2:20,4:40})
    >>> t[:]
    np.ndarray([1,20,3,40])

Column.histogram() will now return a dict with {key1: count1, key2:count2,...} instead of two lists people then have to zip into a dict anyway.

Table.head has been moved to tablite.tools where it belongs.

Deprecated

The following assignment method is DEPRECATED:

    t = Table()
    t[('b','c')] = [ [4,5,6], [7,8,9] ]
    Which then produced the table with two columns:
    t['b'] == [4,5,6]
    t['c'] == [7,8,9]
  • copy_to_clipboard and copy_from_clipboard as pyperclip doesn't seem to be maintained.
  • class method from_dict as Table(columns=dict) now is supported.
  • Insert and append have been removed to discourage operations that are IO-intensive.
  • Column.insert is being removed as it encourages the user to use slow operations. It is better to perform the data manipulation in memory and drop the result into the column using col.extend(....) or col[a:b] = [result].
  • reload_saved_tables. Use t.save(path) and Table.load(path) instead.
  • reset_storage is deprecated as all tables not explicitly saved are considered temporary/volatile.
  • from_dict is deprecated as Table(columns={dict}, ...) is accepted.
  • to_numpy. Default table['name'] returns a numpy array. User should call table['name'].tolist() to get python lists (up to 6x slower) than just retrieving the numpy array.

Everything else remains.