Is fs= deprecated in load_from_disk() as well?

[dconathan]

Describe the bug

The fs= argument was deprecated from Dataset.save_to_disk and Dataset.load_from_disk in favor of automagically figuring it out via fsspec:

datasets/src/datasets/arrow_dataset.py

Lines 1339 to 1340 in 9a7272c

	fs_token_paths = fsspec.get_fs_token_paths(dataset_path, storage_options=storage_options)
	fs: fsspec.AbstractFileSystem = fs_token_paths[0]

Is there a reason the same thing shouldn't also apply to datasets.load.load_from_disk() as well ?

datasets/src/datasets/load.py

Line 1779 in 9a7272c

def load_from_disk(dataset_path: str, fs=None, keep_in_memory: Optional[bool] = None) -> Union[Dataset, DatasetDict]:

Steps to reproduce the bug

n/a

Expected behavior

n/a

Environment info

n/a

[mariosasko]

Hi! Yes, we should deprecate the fs param here. Would you be interested in submitting a PR?

[dconathan]

Hi! Yes, we should deprecate the fs param here. Would you be interested in submitting a PR?

Yeah I can do that sometime next week. Should the storage_options be a new arg here? I’ll look around for anywhere else where fs is an arg.

[albertvillanova]

Closed by #5393.