Skip to content

[Bug]: huggingface/tokenizers warning on mac #89

@gufengc

Description

@gufengc

Steps to Reproduce

on mac, node report
"""
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
"""

Expected Behavior

N/A

Actual Behavior

N/A

Version

main

Environment & Context

  • I'm using the latest version.
  • I have searched existing issues.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions