v0.2.40

What's Changed

• fix: clean parquet dir cache fixture by @deependujha in #474
• Fix: Allow using Machine types in map by @ethanwharris in #473
• 🛠️ Fix: Ensure chunk_bytes in index.json matches actual chunk file size by @bhimrazy in #478
• fix: _get_folder_size fn by @deependujha in #471
• Added boolean serialiser called by litdata.optimise() by @DominiquePaul in #481
• Doc: improve dev doc & add ToDos by @deependujha in #479
• upd: Add hf file download progress and update local file path by @bhimrazy in #484
• fix: segmentation fault error in streaming tokens by @bhimrazy in #485
• Warn user if max_cache_size is less than 25GB in StreamingDataset by @bhimrazy in #489
• fix: Properly assign the chunks to the right worker by @tchaton in #449
• Bump version to 0.2.40 by @bhimrazy in #491

New Contributors

• @ethanwharris made their first contribution in #473
• @DominiquePaul made their first contribution in #481

Full Changelog: v0.2.39...v0.2.40