Skip to content

TimeNet 0.11.0

Choose a tag to compare

@TimoStoff TimoStoff released this 23 Sep 12:02
8f4be1d

TimeNet 0.11.0 replaces the Parquet control tables of TimeF with a DuckDB control plane and a declarative in-memory model. It is a breaking release: TimeF format version 2 replaces version 1, and version 1 directories are not readable. Every registry dataset must be rebuilt with 0.11.0 connectors.

Data model

  • Records hold a Source hierarchy of Signals. Signals are eager or lazy (Signal.from_loader, Signal.from_irregular), and source.select(...) picks Signals to annotate.
  • Every annotated object (TimeFDataset, Record, Source, Signal, Task) implements the SupportsAnnotate protocol with annotate() and annotations.
  • Tasks are declarative: ordered input Records, mixed ordered targets, references to stored Records and annotations, and linear registration with dataset.add_task(task=...).
  • Annotation values are typed: string, integer, float, boolean, or list of strings.

Format

  • control.duckdb (control schema version 1) stores the hierarchy, annotations, tasks, targets, and value chunk locations. The schema is described once in timenet.format.control_schema; audit_control_database() re-checks every relational invariant.
  • Fixed-shape fields are typed columns, so task scope, targets, and annotation values are filterable in SQL. Only free-form metadata is JSON.
  • The manifest reports timef_format_version: 2.

Reader and writer

  • TimeFReader hydrates records one at a time or in batches, reads tasks in one pass, and exposes task_table(), target_table(), and annotation_table() as Arrow tables.
  • TimeFTorchDataset.from_reader() and __getitems__ batch loading.
  • TimeFWriter streams the control plane in typed Arrow batches inside one transaction and validates as it writes.

Removed

Record.time_series, Signal.time_series_id, the TimeSeries alias, Record.add_annotation (use annotate), the Parquet control-plane writers and readers, and the legacy id codec.

Landed as one squash (#361) of #311, #313, #314, #315, #349, #350, #351, #352, #353, #355, #357, #359, and #360.