Very slow read speed on the qemu image

[Eboubaker]

I noticed massive read speed reduction (like 10-20 times) when inside qemu this makes building/pulling images take hours.
Do you have any improvements that can be made here?

I tested with this script https://github.com/thodnev/MonkeyTest/blob/master/monkeytest.py
inside termux i get normal 90MB/s but inside qemu i get 4MB/s.
I tried my best to fix this bottlneck best i got is 10MB/s by using .raw image format instead of Qcow2

[cyberkernelofficial]

You're right, running Docker within QEMU can significantly impact disk read/write speeds, leading to longer build and pull times. This is often due to the virtualization overhead and how QEMU emulates disk access.

While using a .raw disk image instead of QCOW2 can yield some improvement, the core issue often lies in the limited resources available to QEMU. As suggested in this issue, allocating more CPU cores and memory to your QEMU virtual machine can significantly boost performance.

Additionally, ensure that your host system (where you're running QEMU) has sufficient resources available. If the host system is already under heavy load, it will further impact QEMU's performance.

Here are some things you can try:

• Increase CPU Cores: Allocate more CPU cores to the QEMU virtual machine. Experiment to find the optimal number for your system.
• Increase Memory: Provide QEMU with more RAM. Building Docker images can be memory-intensive.
• Experiment with QEMU Settings: Explore QEMU's configuration options related to disk I/O and caching to see if any optimizations can be made.

Let me know if these suggestions help improve your build speeds within QEMU. If you find any specific configurations that work well, please share them with the community so others can benefit!

[Eboubaker]

I feel sad that you had to use AI to write this. However I hope someone can provide configuration that can mitigate the speed issue because this trick is completely useless to run mid-large docker containers.

[lexavey]

AI 💀💀💀 💀💀

[cyberkernelofficial]

I understand your concern about AI use in this project. While I do leverage AI to improve my writing (as a non-native English speaker) and make my communication clearer, I assure you that the core functionality of this repo is not solely reliant on AI-generated code.

I appreciate your feedback on the speed issue. It's definitely a valid concern, and I'm actively working on optimizing the configuration to improve performance for larger Docker containers. I'm open to suggestions and contributions from the community.

Let's work together to make this project as efficient and helpful as possible!

Note: this is AI generated too

[freebrowser1]

Maybe use arm64 architecture instead of x86-64. The latter is emulated on Android and ARM is native speed.

[Eboubaker]

Maybe use arm64 architecture instead of x86-64. The latter is emulated on Android and ARM is native speed.

How can you do that in the terminal exactly I couldn't make it run.
this is what I tried

pkg install qemu-system-arm
qemu-img create -f raw alpine-arch64.img 2G
wget https://dl-cdn.alpinelinux.org/alpine/v3.20/releases/aarch64/alpine-virt-3.20.0-aarch64.iso
qemu-system-arm \
-machine virt \
-m 1024 \
-smp cpus=2 \
-drive if=pflash,format=raw,read-only=on,file=$PREFIX/share/qemu/edk2-aarch64-code.fd \
-netdev user,id=n1,dns=8.8.8.8,hostfwd=tcp::2222-:22 \
-device virtio-net,netdev=n1 \
-drive file=alpine-arch64.img,format=raw,if=virtio \
-cdrom alpine-virt-3.20.0-aarch64.iso \
-nographic

But it just does nothing I waited 30 mins

[freebrowser1]

Typo ?
alpine-arch64 should be alpine-aarch64.

Or maybe try this:

https://hub.docker.com/r/termux/termux-docker/tags

[Eboubaker]

Typo ? alpine-arch64 should be alpine-aarch64.

There is no typo it's the filename of the virtual disk that I have created with qemu-img create -f raw alpine-arch64.img 2G

[freebrowser1]

You can try the link above from hub.docker.com and pick the second entry which is aarch64.
It appears to be edited by 'grimler' who is the maintainer of Termux.

[Eboubaker]

You can try the link above from hub.docker.com and pick the second entry which is aarch64. It appears to be edited by 'grimler' who is the maintainer of Termux.

I don't why are you referencing the Termux Docker image, the purpose of this repository is to run docker inside termux(non rooted android) not the other way around.

[freebrowser1]

Is this not a Docker image for Termux which you can deploy under Termux on a non-rooted Android ?

[Eboubaker]

Is this not a Docker image for Termux which you can deploy under Termux on a non-rooted Android ?

No, Try to read or translate the README.md file https://github.com/cyberkernelofficial/docker-in-termux/blob/main/README.md
The purpose is to install Qemu emulator so that we can run any docker image inside it. (you can't install docker in non-rooted termux)

[Eboubaker]

Ok I have reached to the Mailing list of Qemu https://lists.nongnu.org/archive/html/qemu-arm/2024-06/msg00026.html to get help in regards to running it in aarch64, I have made a mistake by using qemu-system-arm which is aarch32bit I had to use qemu-system-aarch64 it can run arm64 bit.

pkg install qemu-system-aarch64
qemu-img create -f raw alpine-arch64.img 2G
wget https://dl-cdn.alpinelinux.org/alpine/v3.20/releases/aarch64/alpine-virt-3.20.0-aarch64.iso
qemu-system-aarch64 -cpu cortex-a55 \
-machine virt \
-m 1024 \
-smp cpus=2 \
-drive if=pflash,format=raw,read-only=on,file=$PREFIX/share/qemu/edk2-aarch64-code.fd \
-netdev user,id=n1,dns=8.8.8.8,hostfwd=tcp::2222-:22 \
-device virtio-net,netdev=n1 \
-cdrom alpine-virt-3.20.0-aarch64.iso -nographic alpine-arch64.img

I used cpu cortex-a55 here because I checked supported architectures of my phone using CPU-Z app, then I checked the available list using qemu-system-aarch64 -cpu help because I wanted them to match, I don't know if that helps with performance though...

When I did the benchmark again nothing changed... I still get 4Mb/s read speed on the disk which is disappointing. Thanks anyways @freebrowser1 for the idea.

[freebrowser1]

The problem is that the arm64 variant is also emulated (which is slow) rather than virtualized. Most Android devices require hardware virtualization which may require root.

[Eboubaker]

The problem is that the arm64 variant is also emulated (which is slow) rather than virtualized.

The problem is in I/O performance, If we can figure a way to boost disk speed half of the problem is fixed.

Most Android devices require hardware virtualization which may require root.

Consumer smartphones don't have kvm module in their kernel out of the stock, you have to flash a custom rom with root. this is not hard in Pixel devices but I want something that works even without rooting.