Repository navigation
v2.1.0
mef3io 2.1.0
Released 2026-10-31
A minor release that makes writing many channels much faster. Nothing that
worked in 2.0 changes.
What's new
- Write many channels in one call.
Writer.write_int32_many(channels, data, ufact, start_uutc, fs)(integer counts) andWriter.write_many(...)
(float data, NaN gaps, precision decided per channel) write the same stretch
of time for all channels at once. The files are the same as a loop of
write_int32/writecalls would produce (a fresh segment differs only
in its random UUIDs). The channels are encoded and written in parallel on
theWriter'sn_threads, and the disk is waited for once per call
instead of once per channel. Also in C++ (SessionWriter::write_int32_many,
write_float_many), the C ABI (mef3io_writer_write_int32_many) and
MATLAB (Writer.writeInt32Many, integer counts only).
Speed: on synthetic data (64 channels of 10 minutes at 2048 Hz, appended
in 4 rounds, 12 threads, one SSD, one 12-core machine) a loop of
write_int32took 5.2 s andwrite_int32_many0.76 s. Your gain depends on
your disk and channel count; it is largest with many channels and a disk
where waiting for a sync is slow. Numbers and the setup are in the
benchmarks. Writer(flush="close")(opt-in). No wait for the disk during writes;
close()orflush()does one combined wait. If the power is cut while
writing, the whole session can become unreadable, andrecover_session
cannot be relied on to fix it. Use it only for output you rebuild after
any failure. The default,flush="each", is unchanged. It is refused
together withwait_for_disk=Trueand on a session that already existed
when theWriteropened, so earlier recordings are never put at risk.
Batching withwrite_int32_manyalready removes most of the waiting; in the
same synthetic testflush="close"saved only about 10 % more on top of it.
See When is my data on disk?.
Failures in a multi-channel write
Every refusal (a duplicate channel, a different rate or conversion factor
than what is stored, data starting before the stored end) is decided before
any file is touched. A failure before the metadata files are published rolls
every channel back. A failure while they are being published raises
PartialWriteError, whose written_channels and incomplete_channels say
what was written. If a roll-back itself fails, RollbackIncompleteError names
the channels whose files may be torn, never a silent "nothing written": run
validate and recover on them.
Changed
- Writing a fresh segment copies the encoded blocks once less.
Upgrading from 2.0
Nothing is required: no signature, default or file format changed, and the
new calls and option are additive. To get the speed-up, replace a loop of
write_int32 / write over channels with one write_int32_many /
write_many call, and write long runs in chunks of minutes (each call holds
its channels' data and encoded blocks in memory).