v1.3.0

Highlights

Safer concurrent appends (#75)

write_datasource(..., append=...) now takes the datasource's write lock before it reads the existing coordinate for the overlap check. The append is therefore always computed against the version it lands on. Previously, two concurrent appenders could each read the same version and both commit, which could leave duplicated or misaligned records in the store.

When another session is already writing to the datasource, the append now stops before reading or writing anything and raises DatameshWriteError with a message that mentions the existing write session. Pipelines that may start overlapping runs can catch this and exit cleanly.

This needs a gateway that offers the write lock. Against an older gateway the client falls back to the previous behaviour, where the lock is taken at the first write. Plain writes and overwrites are unchanged.

New

• Quantile aggregate (#73): the quantile aggregate operation takes a required q between 0 and 1, e.g. q=0.95 for the 95th percentile. It uses linear interpolation, as in xarray and pandas.

Fixes

• load_datasource on DataFrame datasources (#74) now sends its parameters, with one download file per parameter set.

Full changelog: v1.2.1...v1.3.0