This library cannot reconcile with the react-native onnxuruntime official library: unresolved OrtGetApiBase symbol

[l3utterfly]

1. using libonnxruntime.so from the official react native library:

sherpa-onnx will say cannot resolve "OrtGetApiBase" symbol. I think this is because the official library has versioned API.

2. make the official library use the libonnxruntime.so downloaded from your release page: crashes when using the official library features to run other onnx models

Is there any way we can reconcile this?

[csukuangfj]

make the official library use the libonnxruntime.so downloaded from your release page: crashes when using the official library features to run other onnx models

Do you re-build sherpa-onnx using linbonnxruntime.so from the official react native library?

[l3utterfly]

No, I didn't do that. How can I do that? I'm new to this, please help.

I just downloaded the libs from your release page.

[csukuangfj]

Please see our documentation

https://k2-fsa.github.io/sherpa/onnx/android/index.html

[l3utterfly]

Thanks, I will try that

[csukuangfj]

If you have troubles, please point me to the URL of libonnxruntime.so and I will build one for you.

[l3utterfly]

This is the jettified onnx lib in my transform cache: https://github.com/l3utterfly/jetified-onnxruntime-android-1.17.0

If you can build one I would greatly appreciate it! Thank you!

[csukuangfj]

@l3utterfly

Please see #603

[l3utterfly]

Thank you so much for this! I really appreciate it.

Do you think it's helpful to update the documentation or upload these built libraries to your repo? I believe there are plenty of other users who would like to use both this library and the official onnxruntime library at once.

[csukuangfj]

First, I want to check if it works for you.

If yes, then I think we can add documentation for it and also provide pre-built .so files for it.

[csukuangfj]

We have an on-going issue #433

We need to support react native eventually.

[l3utterfly]

First, I want to check if it works for you.

Yes, this works!

[csukuangfj]

Thanks for verifying it.

Do you have a plan to contribute the React Native support for sherpa-onnx or will your implementation be open sourced?

[l3utterfly]

Sure, I am happy to help where I can! I don't have any custom implementation though.. I just followed your documentation...

[l3utterfly]

I'm only using the TTS portion of sherpa, so I basically followed your documentation on building for android/ios, then wrote a React Native bridge which exposes the one function that TTS needs.

[csukuangfj]

I'm only using the TTS portion of sherpa, so I basically followed your documentation on building for android/ios, then wrote a React Native bridge which exposes the one function that TTS needs.

Will your binding code be open sourced?

[l3utterfly]

This is the binding code:

package com.k2fsa.sherpa.onnx

import android.net.Uri
import com.facebook.react.bridge.Arguments
import com.facebook.react.bridge.Promise
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod
import com.facebook.react.modules.core.DeviceEventManagerModule
import java.io.File
import java.io.FileOutputStream
import java.io.IOException
import java.io.InputStream
import java.nio.charset.StandardCharsets
import java.util.UUID


class Sherpa(reactContext: ReactApplicationContext?) : ReactContextBaseJavaModule(reactContext) {
    private var listenerCount = 0
    private lateinit var tts: OfflineTts

    override fun getName(): String {
        return "Sherpa"
    }

    // this function is called from C++
    private fun callback(samples: FloatArray) {
        //track.write(samples, 0, samples.size, AudioTrack.WRITE_BLOCKING)
    }

    @ReactMethod
    fun initTts(modelDir: String, modelName: String, modelDataDir: String) {

        var ruleFsts: String? = null
        var lexicon: String? = null

        val config = getOfflineTtsConfig(
            modelDir = modelDir!!, modelName = modelName!!, lexicon = lexicon ?: "",
            dataDir = modelDataDir ?: "",
            ruleFsts = ruleFsts ?: ""
        )!!

        if(::tts.isInitialized)
            tts.free();

        tts = OfflineTts(assetManager = null, config = config)
    }

    @ReactMethod
    fun generate(text: String, voiceId: Int, speed: Float, genDir: String, promise: Promise) {
        Thread {
            try {
                val audio = tts.generateWithCallback(
                    text = text,
                    sid = voiceId,
                    speed = speed,
                    callback = this::callback
                )

                // Create a File object for the directory
                val directory = File(genDir)
                // Check if the directory exists, if not, create it
                if (!directory.exists()) {
                    directory.mkdirs()  // This will create the directory including any necessary but nonexistent parent directories.
                }

                val filename = "$genDir/${UUID.randomUUID()}.wav"
                val ok = audio.samples.size > 0 && audio.save(filename)
                if (ok) {
                    // If saving is successful, resolve the promise
                    promise.resolve(filename)
                } else {
                    // If saving failed, reject the promise
                    promise.reject("ERROR", "Failed to generate or save audio.")
                }
            } catch (e: Exception) {
                // Handle any exceptions by rejecting the promise
                promise.reject("ERROR", "Exception occurred: ${e.message}")
            }
        }.start()
    }
}

Currently it only exposes what I need in my application, but it could be a good starting point. It's not encapsulated in its own library or anything.

[l3utterfly]

I'm not sure how useful this is. I did not create a separate package or anything.