Helsinki models error if used in a worker

[escottgoodwin]

I get the following error if I call a Helsinki NLP model with the pipeline in a web worker (as in demo example). It works if called in the main app on the main thread (not ideal). Other hugging face models work fine in the worker.

utils.js:423 Uncaught (in promise) RangeError: Maximum call stack size exceeded
at softmax (utils.js:423:1)
at Function.sample (samplers.js:227:1)
at Function._call (samplers.js:31:1)
at closure (utils.js:544:1)
at Function.generate (models.js:742:1)
at async Function._call (pipelines.js:373:1)

[xenova]

Ah yes, I remember having a similar issue before when loading the large vocabulary (fixed here). So, I'm betting this line is the culprit:

https://github.com/xenova/transformers.js/blob/ee1056bb71249b88f930be10add64a7c633141ae/src/utils.js#L432

It's a simple fix to just replace it with the iterative version of the max function. Let me do that quickly.

[escottgoodwin]

Actually, it seemed to only happen with es-en model. fr-en and de-en, en-fr, fr-en worked just fine. Perhaps the es-en got corrupted on my initial download.

[xenova]

1b9fe0f should fix it. I will make a full release in the next day or so.

[xenova]

Actually, it seemed to only happen with es-en model. fr-en and de-en, en-fr, fr-en worked just fine. Perhaps the es-en got corrupted on my initial download.

Well, the es-en model has a vocabulary size of 64937, while the fr-en only has 59513 tokens (de-en=58100). Perhaps the "limit" is somewhere in between haha. Either way, no worries - since the above changes should fix it. Let me know if you have any other issues. 👍

[xenova]

Published a new version (v1.4.3) which fixes this. Closing the issue now :)