Skip to content

v0.1.4-alpha

Choose a tag to compare

View all tags
@CosminBMemetea CosminBMemetea released this 26 Apr 20:04
· 45 commits to main since this release
v0.1.4
ead8a56

🎉 v0.1.4-alpha – “Data Unleashed!” 🎉

We’re back with another turbo-charged release of canml, packed with powerful new capabilities to make your CAN log workflows smoother than ever! 🚀

✨ What’s new in v0.1.4-alpha?

Multi-DBC magic (load_dbc_files)
Merge any number of .dbc files—optionally prefixing signals to avoid name collisions—so all your bus definitions live under one roof.

Chunked streaming (iter_blf_chunks)
Decode gigabyte-scale BLF logs in bite-sized pandas DataFrame chunks, complete with tqdm progress bars.

Parquet export (to_parquet)
Store your decoded data in fast, compressed Parquet files for downstream analytics or machine-learning pipelines.

Expected-signal injection
Ensure specific signals always appear in your output—with NaN placeholders if they never showed up on the bus.

Message-ID filtering
Only care about a handful of IDs? Pass message_ids={…} to load_blf or filter_ids={…} to iter_blf_chunks and skip the rest.

Improved logging & error handling
Clear, consistent warnings when things go wrong—no more silent skips or missing DBC errors.

Docs & examples updated
Check out the revised README and examples/generate_blf.py for instant hands-on recipes.

>50% tested
New unit & integration tests cover the developed functions.

Quickstart

pip install canml
from canml.canmlio import (
    load_dbc_files,
    iter_blf_chunks,
    load_blf,
    to_csv,
    to_parquet
)

# 1. Merge your DBCs
db = load_dbc_files(["powertrain.dbc", "chassis.dbc"], prefix_signals=True)

# 2. Stream-decode only IDs 0x100 & 0x200 in 20k-row chunks
for idx, chunk in enumerate(iter_blf_chunks(
        blf_path="vehicle.blf",
        db=db,
        chunk_size=20_000,
        filter_ids={0x100, 0x200}
    )):
    to_parquet(chunk, f"shard-{idx:03}.parquet")

# 3. Load a smaller session fully, inject missing signals, uniform timing
df = load_blf(
    blf_path="session0.blf",
    db=db,
    message_ids={0x100, 0x200},
    expected_signals=["EngineData_EngineRPM","BrakeStatus_ABSActive"],
    force_uniform_timing=True
)

# 4. Save to CSV
to_csv(df, "session0_decoded.csv")
print("All done! 🎊")

Huge thanks to everyone who’s tried canml and contributed! If you discover any bugs or dream up new features, please open an issue or submit a PR. Let’s keep powering through those CAN logs together! 🏎️💨

Assets 2
Loading