windows opus-fast-mosestokenizer installation

[tatarmb4s]

Hello! When I try to run the whisper_online_server I get this error:

Exception has occurred: FileNotFoundError
[WinError 2] The system cannot find the file specified
File "S:\AI\Testing\HuggingFace\openai\whisper_streaming\whisper_online.py", line 436, in create_tokenizer
return MosesTokenizer(lan)
^^^^^^^^^^^^^^^^^^^
File "S:\AI\Testing\HuggingFace\openai\whisper_streaming\whisper_online_server.py", line 64, in
online = OnlineASRProcessor(asr,create_tokenizer(tgt_language))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Everything is the default, I just cloned the repo and installed all dependencies. How can I solve this?

I'm on Windows, using py3.11.3 in a .venv.

[tatarmb4s]

I found a temporary solve, in here:
https://stackoverflow.com/questions/55871850/mosestokenizer-issue-winerror-2-the-system-cannot-find-the-file-specified#:~:text=use%20sacremoses%20instead%20of%20moses.%20pip%20install%20-U,%27my%27%2C%20%27name%27%2C%20%27is%27%2C%20%27artem%27%5D%29%29%20for%20complete%20details%20sacremoses

[Gldkslfmsd]

installed all dependencies

Are you sure that you installed opus-fast-mosestokenizer correctly?

And can you confirm that it works well with pip install sacremoses on Windows? No changes in code are needed?

[white1-doggy]

but it doesn't work for the mosestokenizer in sacremoses

AttributeError: 'MosesTokenizer' object has no attribute 'split'

[Gldkslfmsd]

yes, sacremoses does not have sentence segmentation

[white1-doggy]

yes, sacremoses does not have sentence segmentation

so, please tell me how can I slove this question?

[Gldkslfmsd]

resolved -- same reason as #23