Arm platform

[xianhaosui]

Have you considered supporting ARM series processors?

[xianhaosui]

I hope to release Docker images for arm64 and support training

[localai-bot]

⚠️⚠️⚠️⚠️⚠️

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

but.... I can also be funny or helpful 😸 and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me,
I don't support (yet) replying!

⚠️⚠️⚠️⚠️⚠️

Yes, we are planning to support ARM series processors in the future. We will be adding support for ARM-based GPUs in our build scripts and will also work on optimizing the code for better performance on ARM hardware. However, this is a long-term goal and may take some time to implement fully. In the meantime, users with ARM processors can use the software-accelerated builds of OpenBLAS or ClBLAS to run LocalAI models.

Sources:

• https://github.com/go-skynet/LocalAI/tree/master/.env
• https://localai.io/basics/build/index.html
• https://localai.io/basics/getting_started/index.html
• https://localai.io/basics/news/index.html

[Aisuko]

I suggest you build the project on your local arm environment. We support arm see https://localai.io/basics/news/index.html#container-images-1. However, I do not find we have build the container of arm.

[h4ck3rk3y]

@Aisuko are you open for contributions that would build both amd64/arm64 in your CI?

I see your image.yaml is quite verbose and uses buildx; but you choose not to publish arm64

[mudler]

we were. The build time of cross compiling on amd64 was too big to keep it in the CI pipelines.

We need dedicated hardware.

[damianoneill]

we were. The build time of cross compiling on amd64 was too big to keep it in the CI pipelines.

We need dedicated hardware.

@mudler thanks all your efforts.

Would it be possible to limit the build to official tags? That way reducing the need for dedicated hardware? I'm trying to run localai on mac silicon in k3d, failing when pulling the image as amd64 published only.

[vhscom]

I'm trying to run localai on mac silicon in k3d, failing when pulling the image as amd64 published only.

With aarch64/arm64 support users running Apple Silicon will likely be running one of Darwin or Linux. I'd like to see GPU support added for metal builds on Linux before containerized again and then K3D after.

[mudler]

we were. The build time of cross compiling on amd64 was too big to keep it in the CI pipelines.
We need dedicated hardware.

@mudler thanks all your efforts.

Would it be possible to limit the build to official tags? That way reducing the need for dedicated hardware? I'm trying to run localai on mac silicon in k3d, failing when pulling the image as amd64 published only.

probably we could try building only part of the core backends - something we can do quite easily now

[adamcohenhillel]

Hey, is it on the roadmap to support arm?

[mudler]

arm64 builds for core images is now merged on master and will be in the next release - I have to explore still how much we can build of the current image matrix, but I assume we can't as much as it is adding quite some load overall to the CI for the moment.

Update:
master arm64 images are already out master-ffmpeg-core

[pepisito]

trying to do docker pull for arm64 got the message below,

~$ docker pull quay.io/go-skynet/local-ai
Using default tag: latest
latest: Pulling from go-skynet/local-ai
no matching manifest for linux/arm64/v8 in the manifest list entries

Not sure there.

It needs a tag and that works.

ocker pull quay.io/go-skynet/local-ai:master-ffmpeg-core

arm64 builds for core images is now merged on master and will be in the next release - I have to explore still how much we can build of the current image matrix, but I assume we can't as much as it is adding quite some load overall to the CI for the moment.

Update: master arm64 images are already out master-ffmpeg-core

[xianhaosui]

Received, thank you   腾龙云开发 ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "pepe ***@***.***>; 发送时间: 2024年6月20日(星期四) 凌晨0:59 收件人: ***@***.***>; 抄送: ***@***.***>; ***@***.***>; 主题: Re: [mudler/LocalAI] Arm platform (Issue #1249) trying to do docker pull for arm64 got the message below, ~$ docker pull quay.io/go-skynet/local-ai Using default tag: latest latest: Pulling from go-skynet/local-ai no matching manifest for linux/arm64/v8 in the manifest list entries Not sure there. arm64 builds for core images is now merged on master and will be in the next release - I have to explore still how much we can build of the current image matrix, but I assume we can't as much as it is adding quite some load overall to the CI for the moment. Update: master arm64 images are already out master-ffmpeg-core — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[mudler]

only images core image (ffmpeg-core) is supported with the ARM64 architecture as for now. Now we also have arm64 binaries as well. So I'm closing this card as it just needs follow-ups (for instance, to support all the image combinations we currently support on amd64)