2023.6.dev0 - Breaking release with ~6x speedup.
Pre-releaseThis release 2023.6.dev0 addresses #61 with a significant number of changes and features ~6x speedup.
For a quick summary view the tutorial
Changes
A new Config-class available in config.py, which sets:
- config.page_size = 1_000_000
- config.workdir =
tempfile.gettempdir()
The workdir uses ///pages for managing pages, so that different python processes don't interact with the same storage location. Previously two python processes would enter conflict on the same tablite.h5-file.
Table now accepts the keyword columns as a dict in init:
t = Table(columns={'b':[4,5,6], 'c':[7,8,9]})`
Table now accepts header/data combinations in init:
t = Table(header=['b','c'], data=[[4,5,6],[7,8,9]])`
With these features it is no longer necessary to write:
t = Table
t['b'] = [4,5,6]
t['c'] = [7,8,9]Preferred approach for subclassing tables is:
class MyTable(tablite.Table):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.x = kwargs.get("x", 42) # <== special variable required on MyTable.
def copy(self):
# tablite.Table implements
# new = cls(); # self.x is now 42 !!!
# for name,column in self.items():
# new[name] = column
# MyTable therefore implements:
cp = super(MyTable, self).copy()
cp.x = self.x # updating to real x.
return cpReplace is refactored from using a single value to a mapping,
example:
>>> t = Table(columns={'A': [1,2,3,4]})
>>> t['A'].replace({2:20,4:40})
>>> t[:]
np.ndarray([1,20,3,40])
Column.histogram() will now return a dict with {key1: count1, key2:count2,...} instead of two lists people then have to zip into a dict anyway.
Table.head has been moved to tablite.tools where it belongs.
Deprecated
The following assignment method is DEPRECATED:
t = Table()
t[('b','c')] = [ [4,5,6], [7,8,9] ]
Which then produced the table with two columns:
t['b'] == [4,5,6]
t['c'] == [7,8,9]copy_to_clipboardandcopy_from_clipboardaspyperclipdoesn't seem to be maintained.- class method
from_dictasTable(columns=dict)now is supported. - Insert and append have been removed to discourage operations that are IO-intensive.
Column.insertis being removed as it encourages the user to use slow operations. It is better to perform the data manipulation in memory and drop the result into the column usingcol.extend(....)orcol[a:b] = [result].reload_saved_tables. Uset.save(path)andTable.load(path)instead.reset_storageis deprecated as all tables not explicitly saved are considered temporary/volatile.from_dictis deprecated as Table(columns={dict}, ...) is accepted.to_numpy. Default table['name'] returns a numpy array. User should calltable['name'].tolist()to get python lists (up to 6x slower) than just retrieving the numpy array.
Everything else remains.