BindingError: Expected null or instance of VectorInt, got an instance of VectorInt

[starpit]

Hello. Great project! I am running into the following error. upcastPointer does a !== on the desired and actual pointer classes... And finds they are not equal. But, as you can see from the error message, the pointer class names do match. Strange?

BindingError: Expected null or instance of VectorInt, got an instance of VectorInt
    at throwBindingError (@mlc-ai_web-llm.js?v=d602f331:17655:17)
    at upcastPointer (@mlc-ai_web-llm.js?v=d602f331:18022:15)
    at RegisteredPointer.nonConstNoSmartPtrRawPointerToWireType [as toWireType] (@mlc-ai_web-llm.js?v=d602f331:18122:21)
    at Object.vecIntToView (@mlc-ai_web-llm.js?v=d602f331:18344:59)
    at Tokenizer2.encode (@mlc-ai_web-llm.js?v=d602f331:19625:29)
    at LLMChatPipeline.getInputData (@mlc-ai_web-llm.js?v=d602f331:14706:40)
    at LLMChatPipeline.<anonymous> (@mlc-ai_web-llm.js?v=d602f331:14255:36)
    at Generator.next (<anonymous>)
    at @mlc-ai_web-llm.js?v=d602f331:2255:67
    at new Promise (<anonymous>)

[CharlieFRuan]

Hi! Could you provide a reproducing example?

[starpit]

Here is a screenshot from the debugger.

[starpit]

Hi! Could you provide a reproducing example?

heya. hopefully soon, once we open source (by the way, my team at IBM is working closely with sky lab! i see you are, too?).

this seems to happen when i run multiple chat generations in parallel. i am creating a separate engine for each...

[CharlieFRuan]

by the way, my team at IBM is working closely with sky lab! i see you are, too?

Cool! I will be joining sky lab starting this summer!

i am creating a separate engine for each...

I see. Perhaps could you try using just one engine? You can load two models in the same engine and send request to a specified model. Unfrotunately WebLLM does not have continuous batching support... See this example for more: https://github.com/mlc-ai/web-llm/tree/main/examples/multi-models

[starpit]

Thanks for the guidance. Does this mean it is not currently possible to have two chat generations running concurrently?

[starpit]

ok! once i switched to the web worker based execution, concurrency seems to work. closing this, since there is an alternative/workaround. thanks for the help!

[CharlieFRuan]

Great!