Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/add android support #1130

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

AmSmart
Copy link

@AmSmart AmSmart commented Mar 19, 2025

Description of Changes

  • Updated compile Android to re-enable build and fix pipeline errors
  • Added Android native libs to the CPU backend .nuspec file
  • Updated .targets file to include new Android additions
  • Added an extra condition to skip custom DLL resolution on Android because Android auto resolves libs based on generated apk
  • Add Maui test Project (only Android enabled at the moment, will enable iOS in future PR)

Please let me know if any updates are required.

@AmSmart
Copy link
Author

AmSmart commented Mar 19, 2025

cc: @martindevans

@martindevans
Copy link
Member

Thanks for the hard work @AmSmart! I'll try to find some time over the weekend to give this a proper in depth review.

Looking at the CI failures, it looks like some new workloads might need to be installed to compile the project. That'll require some changes in this file I would guess.

@amirvenus
Copy link

Is it CPU inference only or can we have GPU acceleration as well?

@AmSmart
Copy link
Author

AmSmart commented Mar 21, 2025

It's only CPU in this PR.

As far as I know, Mobile GPU inference is still quite slow at the moment. Saw it was even slower than CPU inference in some cases. Besides, it takes a bit of effort building CLBlast from source and trying to get that to work with the Android builds so I didn't consider it.

<file src="runtimes/deps/osx-arm64/libllama.dylib" target="runtimes\osx-arm64\native\libllama.dylib" />
<file src="runtimes/deps/osx-arm64/libllava_shared.dylib" target="runtimes\osx-arm64\native\libllava_shared.dylib" />

<file src="runtimes/deps/android-arm64-v8a/libllama.so" target="runtimes\android-arm64-v8a\native\libllama.so" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can these be split into a separate Backend package for Android (e.g. Backend.CPU.Android)?

@martindevans
Copy link
Member

I've given this a deeper look, overall it looks good!

Left to resolve:

  • Fix those CI errors (workloads must be installed: maui-android)
  • Change nuget package to split out Android binaries from default ones
  • Merge conflict
    • If you're splitting up the nuget this should be easy, just revert all changes to the backend.CPU.nuspec
  • Test run (actually just found this run, so it's done)

@AmSmart
Copy link
Author

AmSmart commented Mar 23, 2025

Great, thanks for the review. Will follow up with the requested changes later today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants