Add LiteRT-LM LoRA adapter support#206
Open
leehack wants to merge 7 commits into
Open
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #206 +/- ##
=======================================
Coverage 80.59% 80.59%
=======================================
Files 85 85
Lines 11407 11408 +1
=======================================
+ Hits 9193 9194 +1
Misses 2214 2214
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
Chat app preview deployed for
|
There was a problem hiding this comment.
Pull request overview
This pull request clarifies LiteRT-LM LoRA adapter limitations in both user-facing docs and the LiteRT-LM backend’s runtime/load-time error reporting, specifically tying the failures to the pinned litert-lm-native@v0.13.1 public C ABI lacking LoRA setters.
Changes:
- Documented that LoRA remains supported via llama.cpp/GGUF backends while native
.litertlmrejectsModelParams.lorasand runtime LoRA operations due to the pinnedv0.13.1C ABI surface. - Updated LiteRT-LM backend error messages to be actionable (include “public C ABI” + pinned version) and reused the same message for runtime LoRA calls.
- Pinned unit tests to assert the improved LoRA error messaging.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| website/docs/platforms/support-matrix.md | Expands LiteRT-LM limitations to explicitly call out LoRA rejection and the v0.13.1 C ABI reason. |
| website/docs/guides/lora-adapters.md | Adds platform guidance: LoRA works on llama.cpp/GGUF; LiteRT-LM .litertlm currently rejects LoRA due to ABI limitations. |
| website/docs/changelog/recent-releases.md | Adds 0.7.2 highlights describing the clarified LiteRT-LM LoRA diagnostics. |
| test/unit/backends/litert_lm/litert_lm_service_test.dart | Strengthens assertions to require the actionable LoRA error substrings (public C ABI, v0.13.1, backend guidance). |
| README.md | Updates LiteRT-LM limitations section to clarify LoRA support scope and the pinned ABI limitation. |
| lib/src/backends/litert_lm/litert_lm_service.dart | Introduces a shared detailed LoRA unsupported message and appends it to load-time validation errors when loras are provided. |
| CHANGELOG.md | Adds 0.7.2 entry describing improved LiteRT-LM LoRA diagnostics and documentation updates. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
# Conflicts: # CHANGELOG.md
leehack
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.litertlmLoRA support for one LiteRT-LM adapter at scale1.0ModelParams.lorasandengine.setLora(...)/removeLora(...)/clearLoras()through the LiteRT-LM service on native targetslitert_lm_session_config_set_lora_file, with an explicit unsupported-runtime error when older native bundles do not export the symbol## UnreleasedchangelogsNative runtime dependency
This PR requires a
litert-lm-nativeruntime that exportslitert_lm_session_config_set_lora_file.Companion native runtime PR: leehack/litert-lm-native#7
Validation
dart analyze lib/src/backends/litert_lm/litert_lm_runtime.dart lib/src/backends/litert_lm/litert_lm_runtime_stub.dart lib/src/backends/litert_lm/litert_lm_service.dart tool/litert_lm_engine_smoke.dart test/unit/backends/litert_lm/litert_lm_runtime_test.dart test/unit/backends/litert_lm/litert_lm_service_test.dart test/unit/backends/litert_lm/litert_lm_backend_test.dart test/unit/backends/litert_lm/worker_test.dart test/unit/backends/native/native_backend_test.dartdart test test/unit/backends/litert_lm/litert_lm_runtime_test.dart test/unit/backends/litert_lm/litert_lm_service_test.dart test/unit/backends/litert_lm/litert_lm_backend_test.dart test/unit/backends/litert_lm/worker_test.dart test/unit/backends/native/native_backend_test.dartdart test test/unit/backends/litert_lm/worker_test.dart./tool/docs/validate_links.shgit diff --checkLLAMADART_LITERT_LM_LIB_DIR=/opt/UnitySrc/personal/llama/litert-lm-native-lora-support/bin/macos/arm64 dart run tool/litert_lm_engine_smoke.dart /private/tmp/litert-lm-v0.13.1-src/runtime/testdata/test_lm.litertlm cpu "What is 2+2? Answer only with the number." 8 256ModelParams.loras:LITERT_LM_LORA=/private/tmp/litert-lm-v0.13.1-src/runtime/testdata/test_lora_rank32_f16_all_ones.tflite dart run tool/litert_lm_engine_smoke.dart ...engine.setLora(...):LITERT_LM_LORA_MODE=set LITERT_LM_LORA=/private/tmp/litert-lm-v0.13.1-src/runtime/testdata/test_lora_rank32_f16_all_ones.tflite dart run tool/litert_lm_engine_smoke.dart ...Notes
The macOS smoke used upstream real LiteRT-LM model/LoRA fixtures, not a production Gemma LoRA adapter. The tiny fixture produced identical decoded text with and without the adapter, so validation relies on positive load/generation plus the negative invalid-adapter control.
Refs #173