Cannot download models locally

[Aksh97]

Hi, I tried to run the commands, which worked perfectly file.

from speechbrain.pretrained import EncoderDecoderASR

asr_model = EncoderDecoderASR.from_hparams(source="speechbrain/asr-crdnn-rnnlm-librispeech")

But when I looked over the model files, the files downloaded are just 158 kbs, and not the full files are downloaded.

Also, when I tried to download the files directly from Files and versions, the ckpt files are downloaded as zip. which cant be used later for inference. IS there any way to resolve it or it has been purposefully made like that.

[Gastron]

The full files are downloaded, but if they're from HuggingFace Hub like here, the full files go the HuggingFace cache, and just symlinks are created, is this what you're actually seeing?

This was just recently discussed in #1055. As this surprises people, maybe we should change it. The HuggingFace cache is the standard location for HuggingFace downloads and the point of a cache is of course to avoid multiple copies of the same files. I'd like to know, though, what do you want to do with the full files - is it not sufficient to have symlinks?

[Aksh97]

Hi @Gastron, thanks for the reply. I wanted to store those files and containerize them so that it does not needs to be downloaded whenever I run the container.
But unlike other repositories, where the full files are downloaded by default, instead of cache. Maybe for speechbrain we can put an arguement symlinks=true, to just add the symlinks and keep that by default. But if we need that to download full files, we can pass the arguement symlinks=false, we can at least download them.

[mravanelli]

Right, I think we should find a solution like that. It looks like users are a bit surprized by the default behavior and we might want to change it.

…

On Thu, Oct 28, 2021, 10:58 PM Akshay Sachdeva ***@***.***> wrote: Hi @Gastron <https://github.com/Gastron>, thanks for the reply. I wanted to store those files and containerize them so that it does not needs to be downloaded whenever I run the container. But unlike other repositories, where the full files are downloaded by default, instead of cache. Maybe for speechbrain we can put an arguement symlinks=true, to just add the symlinks and keep that by default. But if we need that to download full files, we can pass the arguement symlinks=false, we can at least download them. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1070 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEA2ZVXB7JAXDGJ2FFBKBBLUJIEUXANCNFSM5GZNSIFQ> .

[Gastron]

Right, noted.

[Aksh97]

Thanks @Gastron and @mravanelli For the consideration.

[Aksh97]

Hi There, Any further improvement or solution for this or it will take some time to implement the changes?

[mravanelli]

@Gastron, do we want to change the download folder in the end?

[Gastron]

Let's try to briefly discuss in the SpeechBrain core-team meeting today.

[MClarkTurner]

Since, this issue is still open and it has been about 4 months. I also wanted to comment that the current implementation is unintuitive. This is especially egregious given the standard set by PyTorch, upon which speechbrain is built and what an outsider would consider, at a glance, to be standard practice. Furthermore, it was only after stumbling upon this issue board and the comments in #1055 that I was able to resolve changing the location of the cache placement (which is an otherwise undocumented process). For those interested, the solution is the following code:

export HUGGINGFACE_HUB_CACHE=<new_directory>

otherwise the code will default to placing a ./cache/huggingface under your /home directory.

[Adel-Moumen]

Hello,

Is the issue still up? We merged a PR #1817 which is modifying a lot of things related to HuggingFace, Pretrainer and so on. Could you please let me know if now everything is working as intended?

Thanks.