-
Notifications
You must be signed in to change notification settings - Fork 35
feat: MLC support for Android #134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
You have run out of free Bugbot PR reviews for this billing cycle. This will reset on October 2. To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Update: Seems to be working, but need to get real device tomorrow to check! 😂 |
packages/mlc/android/src/main/java/com/callstack/ai/NativeMLCEngineModule.kt
Outdated
Show resolved
Hide resolved
|
For the lack of objection, I am going to shamelessly merge this PR and move on with other features! Hopefully we can unlock some MLC issues in the meantime. Meanwhile, I'll work on some other interesting features! |
This is working, but performance of MLC on Android is questionable and subject to further optimization. On top of that,
top_kand aborting a stream are not supported (will require a follow-up PR to the engine itself).Generally speaking, it all works just fine, but there's UI freeze first time you run the model, likely related to mlc-ai/mlc-llm#1401 (comment)
I have updated parameters on Android to make them slightly more user-friendly and generally cleaned-up models configuration.
Closes #118
Notable changes in the PR: