Skip to content

v0.1.8-alpha

Choose a tag to compare

View all tags
@CosminBMemetea CosminBMemetea released this 30 Apr 05:27
· 16 commits to main since this release
v0.1.8
4c731f2

🎉 v0.1.8 – “Refined Resilience” 🎉

Our biggest overhaul yet: v0.1.8 brings a unified config object, caching, bullet-proof error handling, full metadata support, and more. Everything’s faster, safer, and more flexible—your CAN-ML workflows just leveled up! 🚀

✨ New & Improved in v0.1.8

🛠 Bug Fixes & Stability

Strict DBC validation
Only .dbc extensions accepted, with clear FileNotFound and ParseError messages (#14).

Safe BLFReader cleanup
Wrapped in a context manager to guarantee reader.stop()—no more dangling file handles.

Robust chunk streaming
All exceptions during chunk decode are caught and logged, preventing pipeline crashes.

🎨 Major API Enhancements

CanmlConfig dataclass
Centralized all BLF-loading options (chunk_size, progress_bar, uniform timing, interpolation, sorting, dtype_map) into a single, easy-to-use object.

LRU-cached DBC loader
Cache up to 32 DBC loads, with namespace collision detection and optional prefix_signals for signal-name deduplication.

iter_blf_chunks() revamp
Stream BLF files in memory-safe pandas chunks, filter by message ID or signal name, and show optional progress bars.

load_blf() overhaul
One-call full-file decode with:

  • Automatic missing-signal injection (int signals zero-filled, floats NaN)
  • Configurable uniform timestamp spacing (with raw_timestamp backup)
  • Linear interpolation of gaps if desired
  • Sorting, filtering, and dtype_map enforcement
  • Automatic enum → Categorical conversion
  • Extraction of all custom DBC attributes into df.attrs['signal_attributes']

Enhanced exports

  • to_csv() and to_parquet() now auto-create parent directories
  • Side-dump signal_attributes JSON alongside your data
  • Robust error wrapping and logging for both CSV and Parquet writes

🧪 Testing & Coverage

Brand-new, comprehensive pytest suites for:

DBC loading (load_dbc_files) with caching, collision, and extension checks

Stream decoding (iter_blf_chunks) under filtering, chunking, and error scenarios

Full‐file loading (load_blf) including warnings, injection, sorting, and metadata

CSV/Parquet exports (to_csv/to_parquet) with directory creation and metadata

Achieved 95% branch coverage across all modules

📚 Docs & Examples

Docstrings expanded to cover every new config parameter and feature

Quickstart updated to showcase one-line DBC cache, streaming vs full load, and auto-metadata

🔧 Quickstart

pip install canml==0.1.8
from canml.canmlio import (
    load_dbc_files, load_blf, to_csv, to_parquet, CanmlConfig
)

# 1️⃣ Merge & cache your DBC(s) safely
db = load_dbc_files(["vehicle.dbc", "chassis.dbc"], prefix_signals=True)

# 2️⃣ Configure BLF loading once
cfg = CanmlConfig(
    chunk_size=5000,
    progress_bar=True,
    force_uniform_timing=True,
    interval_seconds=0.02,
    interpolate_missing=True,
    dtype_map={"Engine_RPM": "int32"}
)

# 3️⃣ Full‐file decode with filters, injection, and enums
df = load_blf(
    blf_path="drive.blf",
    db=db,
    config=cfg,
    message_ids={0x100, 0x200},
    expected_signals=["Engine_RPM", "Brake_Active"]
)

print(df.head())
#    timestamp  Engine_RPM  Brake_Active  raw_timestamp
# 0       0.00        8000             0       162523.1
# 1       0.02        8050             0       162523.3

# 4️⃣ Export with metadata side-dump
to_csv(df, "drive_data.csv", metadata_path="drive_data_meta.json")
to_parquet(df, "drive_data.parquet", metadata_path="drive_data_meta.json")

Massive thanks to all contributors and issue reporters—enjoy the smoother, smarter CAN-ML! 🎉🎉

Assets 2
Loading