Noon

Noon is an SDK that makes it easy to infer TensorFlow-Lite and PyTorch(not yet supported) Models.
Since it is implemented based on C++, it can be used on various platforms.

Inference the Tensorflow-Lite model on Android

    // First, Create Noon instance.
    private val nativeLib = Noon()
    private var outputs = mutableListOf() 
    
    // The model(*.tflite) is read from the asset folder.
    val model = getModel(assetManager) 
    val modelSize = model.size
    // InferenceInfo
    val inferenceInfo = InferenceInfo(
        type = InferenceInfo.InferenceType.IMAGE.ordinal,
        mlInfo = MLInfo(
            mode = MLInfo.MlMode.TENSORFLOW_LITE.ordinal,
            delegate = InferenceInfo.InferenceDelegate.CPU.ordinal,
            info = MLInfo.TFLInfo(
                numThread = 4,
                allowFp16PrecisionForFp32 = false
            )
        ),
        model = model,
        modelSize = modelSize,
        input = InferenceInfo.InputInfo(
            // set an input shape for the data to be put as input.
            shape = listOf(image.width, image.height, pixelStride)
        ),
        // If you do not want to transform into a specific shape, you should not pass a value.
        output = InferenceInfo.OutputInfo()
    )
    nativeLib.setup(inferenceInfo)
    // allocate output buffer
    for (i in 0 until nativeLib.getOutputArraySize()) {
        outputs.add(nativeLib.allocFloatBuffer(i, false))
    }
    // inference input
    nativeLib.inferenceUByteArray(input)
    // and get output(result).
    for (i in 0 until nativeLib.getOutputArraySize()) {
        nativeLib.nativeGetFloatArrayOutput(i, outputs[i])
    }

Remaining Work

• Currently only available for Android.
  (The build system will be completely improved so that it can be built with libraries of various platforms such as iOS, Mac, Windows, etc.)
• Need to improve pre- and post-processing functions.
• It is necessary to provide various example apps.
• No PyTorch model inference capabilities. (TensorFlow-Lite library only supports arm64-v8a {6e3666c9})