0.15.0

What's Changed

• Test on Python 3.12 by @tomwhite in #453
• Test NumPy 2 by @tomwhite in #454
• Expose Lithops wait_dur_sec to speed up tests by @tomwhite in #456
• Simplify executor naming by @tomwhite in #457
• Treat warnings as errors when running tests by @tomwhite in #458
• Re-interpret 'fusable' to mean with predecessor operations by @tomwhite in #460
• Don't include hidden arrays when counting max_total_source_arrays by @tomwhite in #463
• Limited implementation of map_overlap by @tomwhite in #462
• Limited implementation of pad by @tomwhite in #461
• Implement roll by @tomwhite in #465
• Fix map_overlap in case of single chunk in overlap dimension by @tomwhite in #466
• Fix case of Lithops 3.4.0 hanging by @tomwhite in #471
• Fix unify_chunks to return regular chunks in all cases by @tomwhite in #470
• Improve test coverage by @tomwhite in #472
• Change development status to beta by @tomwhite in #474
• Top level array namespace by @tomwhite in #473
• Add notebooks for Pangeo examples by @tomwhite in #476

Full Changelog: 0.14.3...0.15.0

0.14.3

What's Changed

• Take broadcast trick into account for array chunk memory by @tomwhite in #447
• Fix Documentation link by @norlandrhagen in #451
• Include j2 templates in the distribution by @tomwhite in #450
• Add links to blog posts by @TomNicholas in #452
• Add a groupby_blockwise function for use in Flox by @tomwhite in #448

New Contributors

• @norlandrhagen made their first contribution in #451

Full Changelog: 0.14.2...0.14.3

0.14.2

What's Changed

• Allow networkx 3.3 now that pydot issue has been fixed by @tomwhite in #445
• Change drop_axis to default to None rather than an empty list in map_blocks by @tomwhite in #446
• Add a groupby_reduction function for use in Flox by @tomwhite in #442

Full Changelog: 0.14.1...0.14.2

0.14.1

What's Changed

• Providing filesystem credentials through storage_options by @balanz24 in #436
• Only use oindex for advanced indexing (int arrays) by @tomwhite in #440
• Deprecate misspelled aggegrate_func in favour of aggregate_func by @tomwhite in #444

New Contributors

• @balanz24 made their first contribution in #436

Full Changelog: 0.14.0...0.14.1

0.14.0

What's Changed

• Handle zero-length slices in index function by @tomwhite in #401
• Run unit tests on Python 3.11 on Linux by @tomwhite in #405
• Update examples to use Python 3.11 by @tomwhite in #406
• Remove unused lazy_full function (replaced by virtual_full) by @tomwhite in #409
• Delegate dtype functions to backend array API by @tomwhite in #410
• Add a 'processes' executor that runs using local processes by @tomwhite in #411
• Mark stack operation as not fusable by @tomwhite in #415
• Track source array names in PrimitiveOperation by @tomwhite in #416
• Fix Modal deprecations by @tomwhite in #417
• Make the new multiple-inputs optimization function the default by @tomwhite in #412
• Add cloudpickle dependency following #411 ('processes' executor) by @tomwhite in #421
• Documentation corrections following optimization improvements by @tomwhite in #423
• Add name property to all executors by @tomwhite in #424
• Build Lithops images for every push to main by @tomwhite in #422
• Make Lithops example names consistent with benchmarks and other executors by @tomwhite in #425
• Delete existing Lithops image before rebuilding by @tomwhite in #427
• Rename block_function to key_function in blockwise by @tomwhite in #426
• Use executor name rather than class name by @tomwhite in #430
• Remove unused sync version of Modal executor by @tomwhite in #431
• Change total_nbytes to total_nbytes_written by @tomwhite in #433
• Fix deprecation warning on logger.warn by @tomwhite in #434

Full Changelog: 0.13.0...0.14.0

0.13.0

What's Changed

• Introduce finalize_dag method in Plan that caches DAG optimization by @tomwhite in #340
• Add VirtualInMemoryArray that keeps small arrays in memory by @tomwhite in #336
• Fix Modal test following #336 by @tomwhite in #341
• Change the DAG to have separate nodes for operations and arrays by @tomwhite in #337
• Allow more control over optimizations applied by @tomwhite in #342
• Handle the case where an array is created by calling map_blocks with no input arrays by @tomwhite in #343
• Add a test for visualizing the quad means computation by @tomwhite in #345
• Show nbytes for each array and total on visualization by @tomwhite in #344
• Fuse multiple inputs by @tomwhite in #346
• Upgrade to sphinx-notes/pages v3 by @tomwhite in #347
• Minor improvements to DAG visualization by @tomwhite in #348
• Remove deprecated code by @tomwhite in #349
• Add generated doc for primitive and core ops to design page by @tomwhite in #351
• Support arbitrary block index functions in blockwise by @tomwhite in #350
• Refactor pipeline to reduce coupling between runtime and primitive layers by @tomwhite in #352
• Implement stack using general_blockwise by @tomwhite in #353
• Implement reshape_chunks using general_blockwise by @tomwhite in #354
• More control over operation fusion by @tomwhite in #355
• Minor memory fixes by @tomwhite in #358
• Use Dask's 'broadcast trick' to save memory for single-valued arrays by @tomwhite in #359
• Add a show_hidden option to visualize by @tomwhite in #360
• Reinstate slow tests by @tomwhite in #361
• Add function_nargs to BlockwiseSpec by @tomwhite in #363
• Add missing api doc by @tomwhite in #364
• Implement partial_reduce and tree_reduce using generalized blockwise by @tomwhite in #365
• Quad means validation test by @tomwhite in #367
• Take allowed_mem into account when fusing primitive operations by @tomwhite in #366
• Fuse operations with different numbers of tasks by @tomwhite in #368
• Show number of tasks prominently on graph visualization by @tomwhite in #371
• Introduce event objects for all callback methods by @tomwhite in #373
• Minor Modal improvements by @tomwhite in #375
• Run tests using JAX as the backend array API (on CPU) by @tomwhite in #374
• Introduce max_total_num_input_blocks as a heuristic to control fusion by @tomwhite in #376
• Rename array_name to name in runtime following #337 by @tomwhite in #377
• Enable backup tasks by default for all executors that support them by @tomwhite in #378
• Remove unused array_names parameter in executors by @tomwhite in #380
• Introduce compute_id for each computation by @tomwhite in #382
• Add a Rich progress bar by @tomwhite in #383
• Change timeline visualization default format to SVG by @tomwhite in #384
• Preserve laziness in fuse_multiple to avoid materializing array blocks in _partial_reduce by @tomwhite in #386
• Fix Modal unit tests following change in #382 by @tomwhite in #387
• Add debug log statements to optimizer to see why nodes are fused by @tomwhite in #388
• Allow optional path in from_zarr and to_zarr by @tomwhite in #391
• Remove CUBED_MODAL_FORCE_BUILD env var … by @tomwhite in #393
• Add introductory slides to docs by @tomwhite in #394
• Use donfig to control default spec by @tomwhite in #392
• small changes to coiled/aws examples by @rsignell in #369
• Skip vecdot test due to zero-size array failures by @tomwhite in #399
• Show both operation ID and name in visualization by @tomwhite in #396
• Add support for minimum_workers in Coiled by @tomwhite in #398
• Create a single set of example scripts that can run on any executor by @tomwhite in #395

New Contributors

• @rsignell made their first contribution in #369

Full Changelog: 0.12.0...0.13.0

0.12.0

What's Changed

• Improve separation between layers by @tomwhite in #314
• Fix Modal deprecation of f.call(...) by @tomwhite in #320
• Disable the Hypthosis deadline when running Array API tests. by @tomwhite in #325
• Allow dtype to be passed to blockwise and reduction functions by @tomwhite in #321
• Array API backend by @tomwhite in #317
• Update examples to use Python 3.9 by @tomwhite in #330
• Re-write arange to use map_direct, which avoids writing a Zarr array by @tomwhite in #332

Full Changelog: 0.11.0...0.12.0

0.11.0

What's Changed

• Introduce VirtualEmptyArray and hide virtual arrays in the DAG visualization by @tomwhite in #290
• Introduce RetryingFunctionExecutor by @tomwhite in #292
• Add support for batching inputs to avoid overwhelming the backend service by @tomwhite in #291
• Support version 2022.12 of the Array API by @tomwhite in #293
• Fix array __abs__ method for complex inputs by @tomwhite in #294
• Be explicit when opening Zarr arrays by @tomwhite in #296
• Fix bug when running with Kerchunk by @tomwhite in #297
• Remove limited implementation of __setitem__ from Array by @tomwhite in #302
• Introduce VirtualFullArray by @tomwhite in #303
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #305
• Fix case where shape is an int for empty by @tomwhite in #306
• Update Coiled example by @tomwhite in #309
• Add nansum and nanmean functions by @tomwhite in #308
• Add moveaxis by @tomwhite in #313
• Update minimum Python version to 3.9 by @tomwhite in #298

Full Changelog: 0.10.0...0.11.0

0.10.0

What's Changed

• Scaling docs by @TomNicholas in #252
• Remove PipelineExecutor classes by @tomwhite in #253
• Surface rechunk intermediates in DAG by @tomwhite in #257
• Add type hints for runtime by @tomwhite in #258
• Fix modal import by @tomwhite in #261
• Add compute_arrays_in_parallel option to Modal and Python async executors by @tomwhite in #263
• Use new merge_chunks function in reduction by @tomwhite in #256
• Remove vendored rechunker API, since we only use the algorithm by @tomwhite in #264
• Add unit tests for virtual_offsets function by @tomwhite in #265
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #266
• Lithops compute arrays in parallel by @tomwhite in #259
• Remove duplicate copy of projected_mem and num_tasks from DAG by @tomwhite in #271
• Allow step size > 1 by @tomwhite in #270
• Fix reserved_mem in create_lazy_zarr_arrays by @tomwhite in #273
• Dask tests by @tomwhite in #269
• Coiled Functions executor by @TomNicholas in #260
• Ensure runtime memory is not less than allowed by @tomwhite in #274
• In-process Python executor should not retry by @tomwhite in #275
• Refactor pipeline mappable by @tomwhite in #272
• Generalise async executor code by @tomwhite in #278
• Dask distributed async executor by @tomwhite in #279
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #283
• Dask check_runtime_memory by @tomwhite in #281
• Fixes for converting strings to bytes by @TomNicholas in #289

Full Changelog: 0.9.0...0.10.0

0.9.0

What's Changed

• Fix Modal deprecated API by @tomwhite in #214
• Modal GCP by @tomwhite in #215
• Add html array repr by @TomNicholas in #216
• Add to pre-commit.ci by @tomwhite in #218
• Skip brew update when setting up GraphViz on mac CI to improve run time by @tomwhite in #227
• Set 'reraise' to True for retries by @tomwhite in #225
• Run mypy in CI by @tomwhite in #229
• Print irregular chunking pattern when error raised by @TomNicholas in #231
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #232
• Allow rechunk to take a dict for chunks by @tomwhite in #226
• Improve Modal backups implementation by @tomwhite in #238
• Test that plan generation scales to very large workloads by @tomwhite in #240
• Don't materialize input iterables to map_unordered by @tomwhite in #242
• Lithops retries and backups by @tomwhite in #217
• Add type hints for primitive functions by @tomwhite in #236
• Add links to Xarray integration and blog post to docs by @tomwhite in #243
• Account for reserved_mem when creating arrays by @tomwhite in #245
• Use np.unravel_index rather than iterating over np.ndindex by @tomwhite in #246
• Add matmul example that validates result by @tomwhite in #244
• Document strong consistency by @tomwhite in #250
• Re-enable rechunker's consolidate_reads by @tomwhite in #249
• Introduce VirtualOffsetsArray that is never materialized by @tomwhite in #248
• Avoid divide by zero from projected mem by @tomwhite in #251

New Contributors

• @pre-commit-ci made their first contribution in #232

Full Changelog: 0.8.0...0.9.0