Repository navigation
v0.9.1
What's Changed
- Add verify_cache param to skip S3 header reads in warmup() by @AlliedToasters in #237
- Add internal profiling/timing mode for performance auditing by @AlliedToasters in #238
- Document custom shard grouping pattern (sort + shuffle=False) by @AlliedToasters in #239
- Parallelize push_dataset metadata scan (#240) by @AlliedToasters in #241
- Add bulk extraction mode for dataset publication by @AlliedToasters in #243
- Fix consolidate_cache() to write to local filesystem by @AlliedToasters in #246
- Add output_uri to consolidate_cache() for S3 output by @AlliedToasters in #248
- Parallelize S3 writes in consolidate_cache() by @AlliedToasters in #250
- Rewrite consolidate_cache() to iterate layer-at-a-time by @AlliedToasters in #253
- Add streaming upload to push_extraction() by @AlliedToasters in #252
Full Changelog: v0.9.0...v0.9.1