thread '<unnamed>' panicked at 'The global thread pool has not been initialized.:

[ruotianluo]

thread '<unnamed>' panicked at 'The global thread pool has not been initialized.: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "R esource temporarily unavailable" }) }', /github/home/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:168:5
Has anyone seen this error.

[NianzuMa]

tokenizer.train(files=paths, vocab_size=vocab_size, min_frequency=5, special_tokens=[
            "<s>",
            "<pad>",
            "</s>",
            "<unk>",
            "<mask>",
        ])

I am seeing a similar error when running this python code.

The error message is

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', /Users/runner/work/tokenizers/tokenizers/tokenizers/src/tokenizer/mod.rs:593:29
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5

===Solution===
My silly mistake. The paths of the file do not exist.
But the error message is not intuitive here. see #349

[n1t0]

thread '' panicked at 'The global thread pool has not been initialized.: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "R esource temporarily unavailable" }) }', /github/home/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:168:5
Has anyone seen this error.

This error seems related to the rayon crate, and I'm unable to reproduce it. Can you provide a minimal example? Ideally in a notebook/collab.

[YulongBonjour]

thread '<unnamed>' panicked at 'The global thread pool has not been initialized.: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "R esource temporarily unavailable" }) }', /github/home/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.7.0/src/registry.rs:168:5
Has anyone seen this error.

I just run into the same error . Have you solved this problem?

[ayushnoori]

I have recently encountered the same error:

pyo3_runtime.PanicException: The global thread pool has not been initialized.: ThreadPoolBuild Error { kind: IOError(Os {code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }

@n1t0, may we reopen this issue, or should we start a new one?

[n1t0]

Can someone provide a way to reproduce this error?

[YulongBonjour]

Can someone provide a way to reproduce this error?

I saw this error when I trained models with 'Autotokenizer.from_pretrained()' and 'torch.distributed.DataParallel.' I think this error might relate to the failure of connection. and I solve this problem by replacing 'Autotokenizer.from_pretrained() ' with a model already downloaded.

[YulongBonjour]

Can someone provide a way to reproduce this error?

I saw this error when I trained models with 'Autotokenizer.from_pretrained()' and 'torch.distributed.DataParallel.' I think this error might relate to the failure of connection. and I solve this problem by replacing 'Autotokenizer.from_pretrained() ' with a model already downloaded.

[ayushnoori]

I have struggled to reproduce this error as well. In this specific instance, I was using SimpleTransformers on a virtual machine with the remote computing extensions in VS Code. However, I did not have any explicit connection issues, and have run the identical script over VPN many times before without incident. Not sure why the error appeared during this particular training loop. Explicitly specifying process_count = 8 as a model parameter via ClassificationArgs seemed to resolve the issue (see documentation here), but it is unclear to me whether this is causation and not correlation. TL;DR, the error vanished as mysteriously as it arrived.

[ChengchengDu]

I also encountered this problem, but I created a new file and then copied the code and the problem was solved. Very strange

[jeremy-feng]

I reloaded VS Code window. It worked for me!

[yl-jiang]

I reloaded VS Code window. It worked for me!

also worked for me! thanks!

[sadrahkm]

If you are using JetBrains products, you shouldn't every time open a new thread to run the code. Try to only have one thread and run your code through that.