Take namespace into account in caching

[lhoestq]

Loading a dataset "username/dataset_name" hosted by a user on the hub used to cache the dataset only taking into account the dataset name, and ignorign the username. Because of this, if a user later loads "dataset_name" without specifying the username, it would reload the dataset from the cache instead of failing.

I changed the dataset cache and module cache mechanism to include the username in the name of the cache directory that is used:

~/.cache/huggingface/datasets/username/dataset_name for the data
~/.cache/huggingface/modules/datasets_modules/datasets/username/dataset_name for the python files

EDIT: actually using three underscores:
~/.cache/huggingface/datasets/username___dataset_name for the data
~/.cache/huggingface/modules/datasets_modules/datasets/username___dataset_name for the python files

This PR should fix the issue #2842

cc @stas00

[severo]

We might have collisions if a username and a dataset_name are the same. Maybe instead serialize the dataset name by replacing / with some string, eg __SLASH__, that will hopefully never appear in a dataset or user name (it's what I did in https://github.com/huggingface/datasets-preview-backend/blob/master/benchmark/scripts/serialize.py. That way, all the datasets are one-level deep directories

[julien-c]

IIRC we enforce that no repo id or username can contain ___ (exactly 3 underscores) specifically for this reason, so you can use that string (that we use in other projects)

cc @Pierrci

[severo]

IIRC we enforce that no repo id or username can contain ___ (exactly 3 underscores) specifically for this reason, so you can use that string (that we use in other projects)

out of curiosity: where is it enforced?

[julien-c]

where is it enforced?

Nowhere yet but we should :) feel free to track in internal tracker and/or implement, as this will be useful in the future

[lhoestq]

Thanks for the trick, I'm doing the change :)
We can use
~/.cache/huggingface/datasets/username___dataset_name for the data
~/.cache/huggingface/modules/datasets_modules/datasets/username___dataset_name for the python files

[lhoestq]

Merging, though it will have to be integrated again the refactor at #2986

[severo]

@lhoestq we changed a bit the naming policy on the Hub, and the substring '--' is now forbidden, which makes it available for serializing the repo names (namespace/repo -> namespace--repo). See https://github.com/huggingface/moon-landing/pull/1657 and huggingface/huggingface_hub#545