TTS onnxruntime bug in latest onnx-runtime (1.18)

[l3utterfly]

Tested with the example SherpaOnnxTTS android application.

To reproduce:

1. Build android arm64_v8a (with onnx 1.18, using https://github.com/csukuangfj/onnxruntime-libs/releases/tag/v1.18.0)
2. Copy built JNI into the example tts project
3. Run app
4. Type "hello, how are you"
5. Click generate, once done speaking, click generate again

App crashes with the following error:

[E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running ScatterND node. Name:'/dp/flows.5/ScatterND_9' Status Message: updates tensor should have shape equal to indices.shape[:-1] + data.shape[indices.shape[-1]:]. updates shape: {0}, indices shape: {44,3}, data shape: {1,1,44}

From my tests, it seems the same tts instance cannot generate words/phonemes with the same text. For example, generating hello twice crashes as well. The only workaround is to release the tts instance and re-initialise it.

This error does not happen with onnx 1.17.x

Error log:

[E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running ScatterND node. Name:'/dp/flows.5/ScatterND_9' Status Message: updates tensor should have shape equal to indices.shape[:-1] + data.shape[indices.shape[-1]:]. updates shape: {0}, indices shape: {44,3}, data shape: {1,1,44}
2024-05-30 14:47:10.742  9707-9886  libc++abi               com.k2fsa.sherpa.onnx                E  terminating due to uncaught exception of type Ort::Exception: Non-zero status code returned while running ScatterND node. Name:'/dp/flows.5/ScatterND_9' Status Message: updates tensor should have shape equal to indices.shape[:-1] + data.shape[indices.shape[-1]:]. updates shape: {0}, indices shape: {44,3}, data shape: {1,1,44}
2024-05-30 14:47:10.743  9707-9886  libc                    com.k2fsa.sherpa.onnx                A  Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 9886 (Thread-4), pid 9707 (fsa.sherpa.onnx)
2024-05-30 14:47:10.931  9889-9889  DEBUG                   crash_dump64                         A  Cmdline: com.k2fsa.sherpa.onnx
2024-05-30 14:47:10.931  9889-9889  DEBUG                   crash_dump64                         A  pid: 9707, tid: 9886, name: Thread-4  >>> com.k2fsa.sherpa.onnx <<<
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #01 pc 00000000000c5620  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (abort_message+260) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #02 pc 00000000000c4f14  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (demangling_terminate_handler()+268) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #03 pc 00000000000c4dd0  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (std::__terminate(void (*)())+12) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #04 pc 00000000000de0f0  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (__cxxabiv1::failed_throw(__cxxabiv1::__cxa_exception*)+32) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #05 pc 00000000000de064  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libkaldi-native-fbank-core.so (__cxa_throw+128) (BuildId: 2de69277be0fb53ed671ab2b639b1cd0b287ad92)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #06 pc 00000000000acb70  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (Ort::detail::ThrowStatus(Ort::Status const&)+292) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #07 pc 00000000000e3018  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (Ort::detail::SessionImpl<OrtSession>::Run(Ort::RunOptions const&, char const* const*, Ort::Value const*, unsigned long, char const* const*, unsigned long)+220) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #08 pc 00000000001b0af4  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsModel::Impl::RunVitsPiperOrCoqui(Ort::Value, long, float)+1256) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #09 pc 00000000001b055c  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsModel::Impl::Run(Ort::Value, long, float)+104) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #10 pc 00000000001b0478  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsModel::Run(Ort::Value, long, float)+48) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #11 pc 00000000001ac36c  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsImpl::Process(std::__ndk1::vector<std::__ndk1::vector<long, std::__ndk1::allocator<long> >, std::__ndk1::allocator<std::__ndk1::vector<long, std::__ndk1::allocator<long> > > > const&, int, float) const+536) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #12 pc 00000000001a39b8  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsImpl::Generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, long, float, std::__ndk1::function<void (float const*, int, float)>) const+1492) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #13 pc 00000000001b42ac  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTts::Generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, long, float, std::__ndk1::function<void (float const*, int, float)>) const+172) (BuildId: 43521a8c14e0f8de74c2144204c5848f95dc81e1)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #14 pc 00000000000199ec  /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/lib/arm64/libsherpa-onnx-jni.so (Java_com_k2fsa_sherpa_onnx_OfflineTts_generateWithCallbackImpl+376) (BuildId: da6dfa3ce2c1ea48c2425e7075bc51b5e0bbf067)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #21 pc 00000000000033f4  [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.OfflineTts.generateWithCallback+0)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #26 pc 0000000000001f90  [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity.onClickGenerate$lambda$3+0)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #31 pc 0000000000001a78  [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity.$r8$lambda$GXwRBIBrY-N067P2cW25xsk89bs+0)
2024-05-30 14:47:10.932  9889-9889  DEBUG                   crash_dump64                         A        #36 pc 00000000000018c4  [anon:dalvik-classes3.dex extracted in memory from /data/app/~~TpgzmBOyXlFDizC1I1t1_Q==/com.k2fsa.sherpa.onnx-fvB-xeNk2V0NgCFFDIbwEQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity$$ExternalSyntheticLambda1.run+0)

[csukuangfj]

Please use 1.17.1 instead.

Some ASR models also have issues with onnxruntime > 1.17.1, including 1.18.0

You can see we are still using onnxruntime 1.17.1