5.0.1

Bug fixes

• Fix version string in init.py by @qgallouedec in #8244
• fix conda build by @lhoestq in #8250
• Fix JSON loader schema inference for files starting with a UTF-8 BOM (#8241) by @archievi in #8243
• Support hermes traces by @lhoestq in #8255
• Fix batch(by_column=...) crashing after shard/shuffle/split by @pkooij in #8259
• fix traces streaming by @lhoestq in #8277
• support droid agent traces by @cfahlgren1 in #8263
• Fix CI: commit operation equality (hfh 1.20.0) and pytest parametrize collection error by @Wauplin in #8283
• Fix lance auth by @lhoestq in #8301
• Fix symlink-following arbitrary file write in archive extraction by @AAtomical in #8303
• Bug Fix: Resuming Twice Resets the Dataloader by @francesco-bertolotti in #8295
• Bump fsspec and simpler wds compr by @lhoestq in #8337
• fix: validate Arrow IPC record batches by @XciD in #8350
• Make the dataset fingerprint independent of Arrow chunking by @SuryanshSS1011 in #8339
• Fix casting a nullable LargeList to a different inner type by @vineethsaivs in #8346
• docs: replace AutoFeatureExtractor with AutoImageProcessor in image preprocessing docs by @gautamkishore in #8326
• Fix column drop in Arrow path of axis=1 concatenation by @ebarkhordar in #8342
• Raise on length mismatch in batched IterableDataset.map by @sohumt123 in #8332
• Fix require_storage_embed recursing into require_storage_cast by @vineethsaivs in #8349
• Fix path traversal via metadata file_name in folder-based builders by @Kaif10 in #8325
• Support batched=True in Dataset.to_dict by @vineethsaivs in #8333
• Fix hdf5 external files by @lhoestq in #8355
• remove bad require_storage test by @lhoestq in #8357
• ensure fiels are in repo by @lhoestq in #8356
• Keep flat numeric columns with nulls numeric in numpy format by @ebarkhordar in #8352
• Fix bucket dataset card handling and push metadata accounting by @pjh4993 in #8354
• Fix CSV loader dropping on_bad_lines/encoding_errors on pandas 2.0-2.2 by @ebarkhordar in #8358
• Keep integers on the python read path for fixed-shape ArrayXD columns with nulls by @ebarkhordar in #8363
• Rebatch arrow source before formatting in IterableDataset.filter to fix resume data loss by @ebarkhordar in #8360
• Decode Json() columns in Dataset.to_pandas() by @ebarkhordar in #8344
• fix buckets on windows by @lhoestq in #8369
• Fix DatasetDict.push_to_hub leaving removed splits in the dataset card by @pjh4993 in #8367
• Preserve nullable integer columns in to_json/to_csv/to_sql by @ebarkhordar in #8366

Docs

• docs: fix duplicate "to" in IterableDataset push-to-hub example by @DaoyuanLi2816 in #8252
• Clarify dataset creation vs loading workflows in create_dataset tutorial by @zanvari in #8235

New Contributors

• @DaoyuanLi2816 made their first contribution in #8252
• @zanvari made their first contribution in #8235
• @archievi made their first contribution in #8243
• @pkooij made their first contribution in #8259
• @AAtomical made their first contribution in #8303
• @francesco-bertolotti made their first contribution in #8295
• @XciD made their first contribution in #8350
• @SuryanshSS1011 made their first contribution in #8339
• @vineethsaivs made their first contribution in #8346
• @gautamkishore made their first contribution in #8326
• @ebarkhordar made their first contribution in #8342
• @sohumt123 made their first contribution in #8332
• @Kaif10 made their first contribution in #8325
• @pjh4993 made their first contribution in #8354

Full Changelog: 5.0.0...5.0.1