Repository navigation
dask-ms 0.3.0-alpha.1: call for testing and breaking changes #392
sjperkins
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
dask-ms 0.3.0-alpha.1 is on PyPI. It replaces python-casacore with arcae in the CASA backend, and moves the experimental zarr backend to zarr-python 3. We'd like downstream projects to test it before 0.3.0 final, which pip will install automatically for any project that doesn't cap
dask-ms<0.3.Try it
pip install --pre "dask-ms==0.3.0a1"pip and uv skip pre-releases unless asked. The alpha won't reach your users until you opt in.
Why
python-casacore is neither thread-safe nor able to release the GIL. dask-ms used to work around this by running all I/O for a table on one thread. arcae releases the GIL and reads through several independent table instances, so CASA reads and writes now run in parallel across dask threads.
Breaking changes
casacore(orpyrap) and relied on dask-ms to install it, addpython-casacoreto your own dependencies. arcae statically links its own casacore, so it can be imported alongside python-casacore.daskms.TableProxyis gone, replaced bydaskms.CasaTable.table_proxy=Truenow returns aCasaTablewrapping an arcaeTable, not a python-casacore-like proxy. Code that calledgetcol/putcol/submiton the returned proxy needs updating.daskms.optimisation(cached_array,inlined_array),daskms.table_proxy,daskms.table_executoranddaskms.ordering.zarr,katdalandcompleteextras now requirezarr>=3.1.3. dask-ms still writes the zarr v2 format, but string columns are now stored asVariableLengthUTF8. Stores with string columns written by dask-ms 0.2.x can't be read and raise an informative error.Additions
xds_from_table/xds_from_mstake an optionalepoch. Calls with the same table, grouping, indexing,taql_whereandepochshare one read of the table's row structure. The default (a fresh random epoch) keeps the old behaviour.taql_wherebehaves as before: dask-ms still uses Python-style (0-based) TaQL indexing.Reporting problems
Reply here for questions. Please open an issue for bugs and regressions, including your dask-ms, arcae and dask versions. The full changelog is in HISTORY.rst.
All reactions