Cache backward compatibility with 2.15.0

[lhoestq]

...for datasets without scripts

It takes into account the changes in cache from

• Lazy data files resolution and offline cache reload #6493: switch to config/version/commit_sha schema
• Refactor dill logic #6454: fix DataFilesDict keys ordering when hashing

requires #6493 to be merged

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[polinaeterna]

it's hard to tell if this works as expected without a test but i guess it's not trivial to implement such a test.

i tried to reproduce locally (with this branch merged into the lazy-resolve-and-cache-reload) and it didn't work.
I run:

 ds = load_dataset("polinaeterna/audiofolder_two_configs_in_metadata", "v2", data_files="v2/train/*") 

and i got this in the cache:

v2-374bfde4f55442bc/
└── 0.0.0
    ├── 5a2339ad2bb7caf6a6daf2f213204e3ac03a13a5  # - from this pr
    │   ├── audiofolder_two_configs_in_metadata-train.arrow
    │   └── dataset_info.json
    ├── 5a2339ad2bb7caf6a6daf2f213204e3ac03a13a5_builder.lock
    ├── 5a2339ad2bb7caf6a6daf2f213204e3ac03a13a5.incomplete_info.lock
    ├── 7896925d64deea5d  # from 2.15.0
    │   ├── audiofolder_two_configs_in_metadata-train.arrow
    │   └── dataset_info.json
    ├── 7896925d64deea5d_builder.lock
    └── 7896925d64deea5d.incomplete_info.lock

so the first hash (the top-level dir v2-374bfde4f55442bc) matches but the second (after version) doesn't.
maybe i did something wrong though.

also i'm not sure if this is worth too much effort, maybe nobody notices if their datasets will be generated again :D idk

[lhoestq]

I just pushed a fix, it should work just fine now :)

[polinaeterna]

that worked!
just in builder.py:1288 _has_legacy_cache() method should also be renamed,
self._has_legacy_cache() -> self._check_legacy_cache() i guess

[github-actions]

Show benchmarks

PyArrow==8.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.004798 / 0.011353 (-0.006555)	0.003203 / 0.011008 (-0.007805)	0.062247 / 0.038508 (0.023738)	0.029906 / 0.023109 (0.006797)	0.259370 / 0.275898 (-0.016528)	0.276084 / 0.323480 (-0.047396)	0.002910 / 0.007986 (-0.005076)	0.002364 / 0.004328 (-0.001964)	0.048080 / 0.004250 (0.043830)	0.041168 / 0.037052 (0.004116)	0.259833 / 0.258489 (0.001343)	0.289882 / 0.293841 (-0.003959)	0.026790 / 0.128546 (-0.101756)	0.010336 / 0.075646 (-0.065311)	0.209628 / 0.419271 (-0.209643)	0.035080 / 0.043533 (-0.008452)	0.256278 / 0.255139 (0.001139)	0.279502 / 0.283200 (-0.003697)	0.019755 / 0.141683 (-0.121928)	1.121552 / 1.452155 (-0.330602)	1.174360 / 1.492716 (-0.318356)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.093510 / 0.018006 (0.075504)	0.302065 / 0.000490 (0.301575)	0.000214 / 0.000200 (0.000014)	0.000043 / 0.000054 (-0.000011)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.017652 / 0.037411 (-0.019759)	0.060512 / 0.014526 (0.045986)	0.072441 / 0.176557 (-0.104115)	0.118058 / 0.737135 (-0.619078)	0.072657 / 0.296338 (-0.223682)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.283949 / 0.215209 (0.068740)	2.803275 / 2.077655 (0.725620)	1.527353 / 1.504120 (0.023233)	1.408176 / 1.541195 (-0.133019)	1.375335 / 1.468490 (-0.093155)	0.546426 / 4.584777 (-4.038351)	2.402210 / 3.745712 (-1.343502)	2.765879 / 5.269862 (-2.503982)	1.703722 / 4.565676 (-2.861955)	0.062669 / 0.424275 (-0.361606)	0.005006 / 0.007607 (-0.002601)	0.337941 / 0.226044 (0.111897)	3.385494 / 2.268929 (1.116566)	1.817360 / 55.444624 (-53.627264)	1.548594 / 6.876477 (-5.327883)	1.548610 / 2.142072 (-0.593463)	0.630188 / 4.805227 (-4.175040)	0.117079 / 6.500664 (-6.383585)	0.042077 / 0.075469 (-0.033392)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	0.941606 / 1.841788 (-0.900182)	11.226277 / 8.074308 (3.151969)	10.118005 / 10.191392 (-0.073387)	0.130408 / 0.680424 (-0.550015)	0.014419 / 0.534201 (-0.519782)	0.284812 / 0.579283 (-0.294471)	0.266951 / 0.434364 (-0.167413)	0.322251 / 0.540337 (-0.218087)	0.415014 / 1.386936 (-0.971922)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.005192 / 0.011353 (-0.006161)	0.003028 / 0.011008 (-0.007980)	0.048322 / 0.038508 (0.009814)	0.030550 / 0.023109 (0.007441)	0.264360 / 0.275898 (-0.011538)	0.289544 / 0.323480 (-0.033936)	0.004053 / 0.007986 (-0.003933)	0.002480 / 0.004328 (-0.001848)	0.048215 / 0.004250 (0.043964)	0.044208 / 0.037052 (0.007156)	0.263943 / 0.258489 (0.005454)	0.297648 / 0.293841 (0.003807)	0.029315 / 0.128546 (-0.099231)	0.010533 / 0.075646 (-0.065114)	0.057021 / 0.419271 (-0.362251)	0.053751 / 0.043533 (0.010218)	0.265153 / 0.255139 (0.010014)	0.284988 / 0.283200 (0.001788)	0.018459 / 0.141683 (-0.123224)	1.225657 / 1.452155 (-0.226498)	1.195737 / 1.492716 (-0.296979)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.093030 / 0.018006 (0.075024)	0.301022 / 0.000490 (0.300533)	0.000228 / 0.000200 (0.000028)	0.000052 / 0.000054 (-0.000002)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.022073 / 0.037411 (-0.015339)	0.075912 / 0.014526 (0.061386)	0.087628 / 0.176557 (-0.088929)	0.125607 / 0.737135 (-0.611529)	0.088568 / 0.296338 (-0.207770)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.303482 / 0.215209 (0.088273)	2.965987 / 2.077655 (0.888333)	1.615273 / 1.504120 (0.111153)	1.482851 / 1.541195 (-0.058344)	1.562627 / 1.468490 (0.094137)	0.563626 / 4.584777 (-4.021151)	2.448741 / 3.745712 (-1.296971)	2.761006 / 5.269862 (-2.508855)	1.711242 / 4.565676 (-2.854434)	0.064593 / 0.424275 (-0.359682)	0.005044 / 0.007607 (-0.002563)	0.354131 / 0.226044 (0.128087)	3.511698 / 2.268929 (1.242770)	1.951087 / 55.444624 (-53.493538)	1.682171 / 6.876477 (-5.194305)	1.666330 / 2.142072 (-0.475742)	0.654880 / 4.805227 (-4.150347)	0.118544 / 6.500664 (-6.382120)	0.040753 / 0.075469 (-0.034717)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	0.967771 / 1.841788 (-0.874017)	12.017277 / 8.074308 (3.942969)	10.624947 / 10.191392 (0.433555)	0.128834 / 0.680424 (-0.551590)	0.015739 / 0.534201 (-0.518462)	0.285906 / 0.579283 (-0.293377)	0.273659 / 0.434364 (-0.160705)	0.324044 / 0.540337 (-0.216293)	0.419469 / 1.386936 (-0.967467)