Bump datasets from 3.2.0 to 3.4.1

[dependabot]

Bumps datasets from 3.2.0 to 3.4.1.

Release notes

Sourced from datasets's releases.

3.4.1

Bug Fixes

• Fix data_files filtering by @​lhoestq in huggingface/datasets#7459

Full Changelog: huggingface/datasets@3.4.0...3.4.1

3.4.0

Dataset Features

• Faster folder based builder + parquet support + allow repeated media + use torchvideo by @​lhoestq in huggingface/datasets#7424

  • /!\ Breaking change: we replaced decord with torchvision to read videos, since decord is not maintained anymore and isn't available for recent python versions, see the video dataset loading documentation here for more details. The Video type is still marked as experimental is this version

  from datasets import load_dataset, Video
  dataset = load_dataset("path/to/video/folder", split="train")
  dataset[0]["video"]  # <torchvision.io.video_reader.VideoReader at 0x1652284c0>

  • faster streaming for image/audio/video folder from Hugging Face
  • support for metadata.parquet in addition to metadata.csv or metadata.jsonl for the metadata of the image/audio/video files

• Add IterableDataset.decode with multithreading by @​lhoestq in huggingface/datasets#7450

  • even faster streaming for image/audio/video folder from Hugging Face if you enable multithreading to decode image/audio/video data:

  dataset = dataset.decode(num_threads=num_threads)

• Add with_split to DatasetDict.map by @​jp1924 in huggingface/datasets#7368

General improvements and bug fixes

• fix: None default with bool type on load creates typing error by @​stephantul in huggingface/datasets#7426
• Use pyupgrade --py39-plus by @​cyyever in huggingface/datasets#7428
• Refactor string_to_dict to return None if there is no match instead of raising ValueError by @​ringohoffman in huggingface/datasets#7435
• Fix small bugs with async map by @​lhoestq in huggingface/datasets#7445
• Fix resuming after ds.set_epoch(new_epoch) by @​lhoestq in huggingface/datasets#7451
• minor docs changes by @​lhoestq in huggingface/datasets#7452

New Contributors

• @​stephantul made their first contribution in huggingface/datasets#7426
• @​cyyever made their first contribution in huggingface/datasets#7428
• @​jp1924 made their first contribution in huggingface/datasets#7368

Full Changelog: huggingface/datasets@3.3.2...3.4.0

3.3.2

Bug fixes

• Attempt to fix multiprocessing hang by closing and joining the pool before termination by @​dakinggg in huggingface/datasets#7411
• Gracefully cancel async tasks by @​lhoestq in huggingface/datasets#7414

Other general improvements

... (truncated)

Commits

• f742152 release: 3.4.1 (#7460)
• e7f1def Fix data_files filtering (#7459)
• 6f7cd01 set dev version (#7454)
• 14fb15a release: 3.4.0 (#7453)
• 97ff626 minor docs changes (#7452)
• e8ee24a Fix resuming after ds.set_epoch(new_epoch) (#7451)
• 7ad7379 Add IterableDataset.decode with multithreading (#7450)
• f09db01 Fix small bugs with async map (#7445)
• 67ffdfb Refactor string_to_dict to return None if there is no match instead of ra...
• f693f4e Add with_split to DatasetDict.map (#7368)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)