Skip to content

Native dylibs native-v0.18.0-e

Choose a tag to compare

@DenisovAV DenisovAV released this 12 Oct 16:48

native-v0.18.0-d with libLiteRtLm rebuilt in all seven archives. Every other library is byte-identical to native-v0.18.0-d, with two exceptions on Windows:

  • StreamProxy.dll was relinked; only its link timestamps differ.
  • LiteRtDispatch.dll is -d's file, carried over byte for byte.

EmbeddingGemma 2 on Linux and Windows (patch_c_api.sh §13). EmbeddingGemma 2 bundles keep their weights in TFLiteWeights sections, which LiteRT-LM hands to the runtime through litert::Options::SetExternalWeightScopedFile. That call passes a C++ std::unordered_map. Our libLiteRtLm is built with libstdc++ on Linux and the MSVC STL on Windows, while Google's prebuilt libLiteRt is built with libc++ (std::__u), so the two read the map with different layouts:

  • On Linux, embedding_engine_create failed, after an openat("tflite_weights") that returned ENOENT.
  • On Windows, the app crashed with an access violation inside libLiteRt.dll.

This build no longer passes the map on Linux and Windows. Instead, libLiteRtLm rewrites each model's external-buffer group to the bundle's own path (Linux: /proc/self/fd/<dup>; Windows: the final path, or its 8.3 alias if the path is not ASCII) and shifts the buffer offsets, so the runtime maps the weights itself through its file-path branch.

  • libLiteRt stays Google's prebuilt. Rebuilding it from source fixed the weights but broke the WebGPU accelerator.
  • Apple and Android, where both libraries use the same STL, compile the change out.
  • LLM bundles have no TFLiteWeights sections and are unaffected.

Two new exports (patch_c_api.sh §14): litert_lm_embedding_response_get_input_length and litert_lm_embedding_response_get_truncated_length. They return the token count of an embedding input and the length it was truncated to, and -1 for a NULL response. They are exported on every platform, the Windows .def included.

Verified with the libraries in these archives:

Platform EmbeddingGemma 2 (embeddinggemma2_test, 22) LLM (litertlm_ffi_test) Tools STT / TTS
Linux x86_64 (GCE, L4 / T4) 22/22 (-d: 2/22) pass (GPU ~110 chunks/s) 6/6 CPU pass
Linux arm64 (GCE Ampere, CPU) 22/22 (-d: 2/22) same results as -d 4/4 public models pass
Linux arm64, Qualcomm QCS8275 (VENTUNO Q) NPU bundles 4/4; CPU bundle matches the golden NPU group passes (Gemma 4 text, vision, audio) — —
Windows x86_64 (GCE, T4) 19/22: the 3 photo-preparation cases need an interactive session for Flutter's decoder (-d: crash) 22/22 GPU, 22/22 CPU 6/6 pass
macOS arm64 22/22 22 + 5 NPU skipped 6/6 pass
iOS (iPhone 17 Pro) 22/22 22 + 5 NPU skipped 6/6 pass
Android (Galaxy S24 Adreno 750, Pixel 8a Mali-G715) 22/22 on both 27/27 on both 6/6 pass
  • Not re-run on -e:
    • Intel NPU: the Lunar Lake box is unreachable. LiteRtDispatch.dll is -d's file.
    • Android Qualcomm NPU.
  • The Qualcomm dispatch libraries for Android and Linux are byte-identical to -d. The Linux Qualcomm NPU path was re-run on -e: Gemma 4 E2B on the QCS8275 HTP decoded at 31.8 tok/s in Burst mode.

checksums_litertlm.txt lists the SHA-256 of every archive; flutter_edge_ai_litertlm's build hook verifies the same values.