[Bug]: Failed to update binfmts when using Docker in Docker

[doragasu]

What happened?

I want to build a custom armbian image in a Gitlab runner. As the Gitlab runner runs a Docker image and the armbian build system on x64 also runs a Docker image, I am using cruizba/ubuntu-dind:latest, that implements the Docker in Docker feature.

Unfortunately, when running the compile.sh script inside the ubuntu-dind container, build fails as follows:

[🌱] Cross arch build [ target arm64 on host amd64 ]
/usr/local/bin/modprobe: 13: ip: not found
[💥] Failed to update binfmts [ update-binfmts --enable qemu-arm ]
[💥] Failed to update binfmts [ update-binfmts --enable qemu-aarch64 ]
[💥] Failed to update binfmts [ update-binfmts --enable qemu-riscv64 ]
[🌱] checking [ arch-test for 'arm64' ]
[🔨]   arm64: not supported on this machine/kernel
[💥] Error 1 occurred in main shell [ at /home/ubuntu/src/wiyo/armbian-base-opi/build/lib/functions/logging/runners.sh:211
    run_host_command_logged_raw() --> lib/functions/logging/runners.sh:211
        run_host_command_logged() --> lib/functions/logging/runners.sh:193
       prepare_host_binfmt_qemu() --> lib/functions/rootfs/qemu-static.sh:94
    prepare_host_noninteractive() --> lib/functions/host/prepare-host.sh:120
                do_with_logging() --> lib/functions/logging/section-logging.sh:81
                   prepare_host() --> lib/functions/host/prepare-host.sh:17
              prepare_host_init() --> lib/functions/main/start-end.sh:64
       main_default_start_build() --> lib/functions/main/start-end.sh:17
          do_with_default_build() --> lib/functions/main/default-build.sh:41
         cli_standard_build_run() --> lib/functions/cli/cli-build.sh:25
        armbian_cli_run_command() --> lib/functions/cli/utils-cli.sh:136
                 cli_entrypoint() --> lib/functions/cli/entrypoint.sh:176
                           main() --> ./compile.sh:50
 ]

How to reproduce?

docker run --privileged --rm -v $PWD:/home/ubuntu/src --entrypoint=/bin/bash -it cruizba/ubuntu-dind

Then, inside the Docker container:

start-docker.sh
cd /home/ubuntu/src
./compile.sh build \
ALLOW_ROOT=yes \
BOARD=orangepi3-lts \
BRANCH=current \
BSPFREEZE=yes \
NETWORKING_STACK=systemd-networkd \
CONSOLE_AUTOLOGIN=no \
RELEASE=bookworm \
BUILD_DESKTOP=no \
BUILD_MINIMAL=yes \
KERNEL_CONFIGURE=no

Branch

main (main development branch)

On which host OS are you running the build script and observing this problem?

Ubuntu 24.04 Noble

Are you building on Windows WSL2?

• Yes, my Ubuntu/Debian/OtherOS is running on WSL2

Relevant log URL

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Hey @doragasu !

Looks like it's your first time interacting with Armbian here on GitHub.

Welcome and thank you for taking the time to report an issue ❤️.

Don't forget to star ⭐ the repo.

[github-actions]

Jira ticket: AR-2672

[EvilOlaf]

Marking as I am not aware that the fw was ever intended to run in a nested env.

[igorpecovnik]

AFAIK we never expected / tested such usage. Totally blindly - try with PREFER_DOCKER=no

[rpardini]

Docker-in-Docker is not supported.

[doragasu]

It's unfortunate docker in docker is not supported. Is there an alternative way to run the builder in docker based environments? I tried also using the armbian based image directly with ALLOW_ROOT, but didn't get it to work.

[igorpecovnik]

It's unfortunate

At some point, we must define boundaries—supporting "everything" simply isn’t feasible, especially when this service is provided entirely free of charge. We aim to avoid the "Market Garden syndrome," where overextending leads to general instability.

Our current scope is already complex and resource-intensive to maintain.

[doragasu]

Yeah, I completely understand you cannot implement everything. But I suppose there is already a way to make this work in CI/CD environments I am not aware of, since you build many images for the supported devices (and I presume those builds are automated). Maybe your CI/CD build system is not Docker based? How does it work?

[igorpecovnik]

Maybe your CI/CD build system is not Docker based? How does it work?

It run in GitHub runners, which are Ubuntu VM instances. Our runners are placed on bare metal and build system make use of Docker internally. tl;dr; it can run native but it can use Docker if installed on that host. In general, Docker is preinstalled. This is a script to prepare bare metal.

As you mentioned GitLab - I don't know much how things works there.

[doragasu]

OK, thanks. I don't know if running VMs is possible with Gitlab CI/CD (AFAIK it can only run docker containers, and then runs a script inside them), but will investigate.

I'm closing this issue, but should I open a feature request for the Docker in Docker feature, or alternatively the ability to run the build script from inside the official armbian Docker images?