"Error 13 initializing the Gpio driver" with .NET 8 in rootless container

[mu88]

Describe the bug

After upgrading my app to .NET 8 and switching to rootless containers (corresponding commit), the following exception occurs when trying to access a Raspi's GPIO pin :

Unhandled exception. System.IO.IOException: Error 13 initializing the Gpio driver.
   at System.Device.Gpio.Drivers.RaspberryPi3LinuxDriver.Initialize()
   at System.Device.Gpio.Drivers.RaspberryPi3LinuxDriver.OpenPin(Int32 pinNumber)
   at System.Device.Gpio.Drivers.RaspberryPi3Driver.OpenPin(Int32 pinNumber)
   at System.Device.Gpio.GpioController.OpenPinCore(Int32 pinNumber)
   at System.Device.Gpio.GpioController.OpenPin(Int32 pinNumber)
   at System.Device.Gpio.GpioController.OpenPin(Int32 pinNumber, PinMode mode)
   at RaspiFanController.Logic.RaspiFanController..ctor(ILogger`1 logger, IOptionsMonitor`1 settings) in /home/runner/work/RaspiFanController/RaspiFanController/RaspiFanController/Logic/RaspiFanController.cs:line 16

Steps to reproduce

1. Clone/fork my repo
2. Call dotnet publish RaspiFanController/RaspiFanController.csproj --os linux --arch arm64 /t:PublishContainer
3. Start a Docker container on a Raspberry Pi 4 with the following docker-compose.yml:

version: '3'
services:
    app:
        environment:
            - LD_LIBRARY_PATH=/opt/vc/lib
            - AppSettings__UpperTemperatureThreshold=70
            - AppSettings__LowerTemperatureThreshold=55
        restart: unless-stopped
        image: mu88/raspifancontroller:latest
        ports:
            - 127.0.0.1:5000:8080
        volumes:
            - /lib:/lib
            - /usr/bin/vcgencmd:/usr/bin/vcgencmd
        devices:
            - "/dev/vchiq"
            - "/dev/gpiomem"

Expected behavior

The GPIO pin can be accessed.

Actual behavior

The following exception occurs:

Unhandled exception. System.IO.IOException: Error 13 initializing the Gpio driver.
   at System.Device.Gpio.Drivers.RaspberryPi3LinuxDriver.Initialize()
   at System.Device.Gpio.Drivers.RaspberryPi3LinuxDriver.OpenPin(Int32 pinNumber)
   at System.Device.Gpio.Drivers.RaspberryPi3Driver.OpenPin(Int32 pinNumber)
   at System.Device.Gpio.GpioController.OpenPinCore(Int32 pinNumber)
   at System.Device.Gpio.GpioController.OpenPin(Int32 pinNumber)
   at System.Device.Gpio.GpioController.OpenPin(Int32 pinNumber, PinMode mode)
   at RaspiFanController.Logic.RaspiFanController..ctor(ILogger`1 logger, IOptionsMonitor`1 settings) in /home/runner/work/RaspiFanController/RaspiFanController/RaspiFanController/Logic/RaspiFanController.cs:line 16

Versions used

• dotnet --info: the project was built using a GitHub Action using .NET 8 and the SDK Container Building Tools
• dotnet --info: the app is running inside a Docker container using .NET 8
• Version of System.Device.Gpio package: 3.0.0
• Version of Iot.Device.Bindings package: 3.0.0

CC: @baronfel (maybe interesting for you as well)

[baronfel]

what if you add privileged: true to the app service?

[mu88]

wow, that was a fast response!

Adding privileged: true to docker-compose.yml seems to have no impact - the error remains the same

[baronfel]

For this you may need to have our tooling change the container user to root. The normal way to do this would be <ContainerUser>root</ContainerUser> in your project file, but there is currently a bug that we will fix in the next patch release of .NET 8. For now, can you try adding the following to the project file?

<ItemGroup>
    <ContainerEnvironmentVariable Include="APP_UID" Value="root" />
</ItemGroup>

[mu88]

Thx, I will try! Some additional questions:

1. How would a solution without using root look like? Is that even possible?
2. Could you please link the other issue (regarding <ContainerUser>root</ContainerUser>) so that I can subscribe to it?

[baronfel]

Sure - the issue is dotnet/sdk-container-builds#520.

As for what a root-less solution for gpio might look like, I don't have a ton of experience with this specific problem. It's The best I could find was this Stack Overflow answer that goes over some of the options - including running the container in privileged mode, mounting a device, or mounting a file system. It look like you've done the device mount, but that didn't work, and you say that overriding the privilege setting didn't work as well.

One thing to try might be using docker directly instead of docker compose just to rule out any problems in the compose layer that may not exist in the purely docker side.

[mu88]

Thank you for your help anyway! Let's see whether the GPIO experts know some additional details :)

[Ellerbach]

[Triage] @mu88 the pattern you can use is to mount and bind the folder from the host (so in our case /dev/gpiomem). See the example here: https://github.com/dotnet/iot/tree/main/samples/led-blink#running-in-containers

See also more on the Docker docs: https://docs.docker.com/storage/volumes/

[mu88]

@Ellerbach can you please elaborate on the difference between your mentioned approach and the one I'm already using in my docker-compose.yml (please see my very first post)? To what I understand, the following line should do that, but maybe I'm missing something:

version: '3'
services:
    app:
        devices:
            - "/dev/gpiomem"

[pgrawehr]

@mu88 I'm not a container expert, but can you try moving the "/dev/gpiomem" to the "volumes" section?

[mu88]

@pgrawehr when using the following docker-compose.yml, the exception Unhandled exception. System.IO.IOException: Error 13 initializing the Gpio driver. is back:

version: '3'
services:
  raspifancontroller:
    container_name: raspifancontroller
    user: app
    environment:
      - LD_LIBRARY_PATH=/opt/vc/lib
      - AppSettings__UpperTemperatureThreshold=70
      - AppSettings__LowerTemperatureThreshold=55
    restart: unless-stopped
    image: mu88/raspifancontroller:latest
    ports:
      - 127.0.0.1:5000:8080
    volumes:
      - /lib:/lib
      - /usr/bin/vcgencmd:/usr/bin/vcgencmd
      - /dev/gpiomem:/dev/gpiomem
    devices:
      - /dev/vchiq

[mu88]

Do you have further ideas? Just let me know if I can provide you more information :)

[pgrawehr]

Can you try without container? Does that work?

[mu88]

Using this release (self-contained .NET 8 app for arm64) on my RPi 4 works without any issues

[pgrawehr]

@mu88 I've never used docker on the RPI, but others have successfully done it. I'm not sure what exactly the problem is, but since the default RaspberryPi3LinuxDriver accesses the GPIO pins via a memory map, it's possible that this doesn't work from a container. You might want to try to use another driver in this case. Since in your setup, performance is not an issue (you don't need to switch the fan on and off thousands of times per second) you can even use the slow SysFsDriver.

Just create the GpioController with a call to:

var controller = new GpioController(PinNumberingScheme.Logical, new SysFsDriver());

or

var controller = new GpioController(PinNumberingScheme.Logical, new LibGpiodDriver());

Side note: We also have a binding for reading out the CPU temperature, so you don't need to make that ugly call to vcgencmd. Just use Iot.Device.CpuTemperature.CpuTemperature.

[mu88]

@pgrawehr thx for pointing me to the way more elegant way of resolving the CPU temperature 😵💪🏻🤩

Regarding Docker: running the app within a container is no problem at all. The problem only arises when switching from using root to a rootless container. It even works with the latest chiseled images, as long as I use the root user.

[Ellerbach]

Regarding Docker: running the app within a container is no problem at all. The problem only arises when switching from using root to a rootless container. It even works with the latest chiseled images, as long as I use the root user.

Stupid question: did you gave the privileges for docker to access /dev/gpiomem ? You still have to make sure that the docker process have the proper rights on it.

[mu88]

Stupid question back 😄: isn't the Docker process the same, no matter which user is used inside the container (root vs. app)? Since it is working with the root user, I would have assumed that the Docker process itself has sufficient permissions

[Ellerbach]

I would have assumed that the Docker process itself has sufficient permissions

It may not have. Give access to the gpiomem so it can make things properly with low privilege. With high priviledges, the story is different!

[mu88]

Learning never stops, I will try this 💪🏻 could you please give me a little guidance on how I can give the Docker user the necessary permissions? Thx a lot!

Please bear with me that my feedback will take 1 month or so because I'm currently not at home.

[Ellerbach]

I would recommend to read couple of articles about this. See for example: https://medium.com/@nielssj/docker-volumes-and-file-system-permissions-772c1aee23ca. I'm quite sure adjusting the permissions should solve your problem! Nothing is urgent for the answer!

[mu88]

Okay, I played a bit with the permissions and can give you an update: for testing purposes, I permitted reading and writing to all users by executing sudo chmod o=rw /dev/gpiomem on the host. Now a rootless container can be successfully started (confirmed via htop which shows the UID 64198) 💪🏻
After executing sudo chmod o-rw /dev/gpiomem on the host (thereby removing the permissions) and restarting the container, the problem is back.

However, simply allowing everybody to read/write /dev/gpiomem feels too permissive to me 🤔 Since /dev/gpiomem is owned by the gpio group, I tried adding the UID 64198 to this group via sudo adduser 64198 gpio which fails as the user (obviously) doesn't exist on the host:
adduser: The user 64198 does not exist.

So is the recommended approach to grant all others read and write permissions? Or create the app user with UID 64198 on the host as well and add it to the gpio group?

CC @richlander (maybe interesting for you as well)

[Ellerbach]

So is the recommended approach to grant all others read and write permissions? Or create the app user with UID 64198 on the host as well and add it to the gpio group?

First, happy, that it was about permissions ;-) There are multiple options to solve the problem. In short, it's about mapping the users from in to outside the container. This gives a good idea how to do it (well, I think): https://serverfault.com/questions/1075488/podman-rootless-container-permissions-for-container-user and most likely this one: https://stackoverflow.com/questions/39397548/how-to-give-non-root-user-in-docker-container-access-to-a-volume-mounted-on-the

[mu88]

I think I'm getting closer, but still not there 🤪

Let me summarize the options according to my current understanding:

1. Assign all users r/w permissions via sudo chmod o=rw /dev/gpiomem on the host and run my container with the app user.
2. Create the app user with UID 64198 on the host as well and add this user to the gpio group.
3. Create a dedicated user for this particular app on the host and add this user to the gpio group.

Options 1 and 2 seem too permissive to me, so I decided to go with option 3:

1. Create new user app_raspi with UID 64200 on the host: sudo adduser --uid 64200 --disabled-password --no-create-home app_raspi
2. Add the newly created user app_raspi to the gpio group: sudo usermod -a -G gpio app_raspi

However, when starting my container with the newly created user, it still fails with the same error. So I checked the following things on the host:

myUser@myRaspi:~ $ grep gpio /etc/group
gpio:x:997:myUser,app_raspi

myUser@myRaspi:~ $ grep app_raspi /etc/passwd
app_raspi:x:64200:64200:,,,:/home/app_raspi:/bin/bash

myUser@myRaspi:~ $ sudo -u \#64200 test -r /dev/gpiomem; echo "$?"
0

myUser@myRaspi:~ $ sudo -u \#64200 test -w /dev/gpiomem; echo "$?"
0

myUser@myRaspi:~ $ ls -lh /dev/gpiomem
crw-rw---- 1 root gpio 245, 0 Jan  6 12:00 /dev/gpiomem

So I started the container with bash as entrypoint:
docker run -it --rm --user=64200 -v /dev/gpiomem:/dev/gpiomem --entrypoint /bin/bash mu88/raspifancontroller:latest
Here's some output as well:

I have no name!@9218798747ed:/app$ whoami
whoami: cannot find name for user ID 64200

I have no name!@9218798747ed:/app$ id
uid=64200 gid=0(root) groups=0(root)

I have no name!@9218798747ed:/app$ test -r /dev/gpiomem; echo "$?"
1

I have no name!@9218798747ed:/app$ test -w /dev/gpiomem; echo "$?"
1

I have no name!@9218798747ed:/app$ test -r RaspiFanController.dll; echo "$?"
0

I have no name!@9218798747ed:/app$ test -x RaspiFanController.dll; echo "$?"
0

I have no name!@9218798747ed:/app$ ls -lh /dev/gpiomem
crw-rw---- 1 root 997 245, 0 Jan  6 11:00 /dev/gpiomem

So from the mentioned, very helpful blog posts I would have assumed that it should now work as app_raspi is a member of the gpio group on the host with r/w permissions and the container runs as 64200 - but I'm still missing something 🤔

[mu88]

I also filed this discussion in a Docker forum

[rimelek]

@Ellerbach

the pattern you can use is to mount and bind the folder from the host (so in our case /dev/gpiomem

That is a character device file, not a folder actually. The --device options makes it possible to "mount" a device properly without using the privileged flag.

@mu88
I answered on the forum, but I do it here too to find it more easily:

Reading the whole issue (and also the topic on the Docker forum), it seems the porblem is the lack of group membership in the container. Setting group membership on the host won't help, neither creating a user on the host. The user can be set by using the --user option and optionally it accepts a group ID as well like --user USER_ID:GROUP_ID. Instead of overiding group memberships in case of the user with the user id exists in the container, --group-add can be used as well to add more groups.

[mu88]

I finally have a working solution: as already mentioned by @rimelek , I had to specify the group ID 997 so that the container can access the GPIO pins.

What did not work with this approach was reading the Raspi's current temperature via vcgencmd. However, as @pgrawehr already mentioned in his reply: the built-in temperature provider Iot.Device.CpuTemperature.CpuTemperature directly parses the file /sys/class/thermal/thermal_zone0/temp without using vcgencmd. With this approach, it works in my scenario as well 💪🏻

This is my final docker-compose.yaml:

version: '3'
services:
  raspifancontroller:
    container_name: raspifancontroller
    user: "64198:997" # group 997 is necessary to access the GPIO pins
    environment:
      - AppSettings__UpperTemperatureThreshold=70
      - AppSettings__LowerTemperatureThreshold=55
    image: mu88/raspifancontroller:latest-chiseled
    ports:
      - 127.0.0.1:5000:8080
    volumes:
      - /sys/class/thermal/thermal_zone0:/sys/class/thermal/thermal_zone0:ro # CpuTemperature needs this
    devices:
      - /dev/gpiomem

With this, I can even use the new chiseled base image.

Thank you all very much for your help, I really appreciated it!

[Ellerbach]

I finally have a working solution

Perfect! And if you're willing to document his with a PR, that would be amazing!

[mu88]

Oh, I'd be glad to do so 😊 where would you like to see this and which content?

[Ellerbach]

Oh, I'd be glad to do so 😊 where would you like to see this and which content?

Amazing! I would say here: https://github.com/dotnet/iot/tree/main/Documentation

We already have bunch of useful articles to set things up properly . So I guess it will be a nice addition.

[mu88]

Hi @Ellerbach 👋🏻 I've created this first draft - would you mind having a look whether it matches your expectations and Microsoft's quality standards? If the draft is mature enough, I'd be happy to file a PR where we could discuss the details that might need to be adapted.

[Ellerbach]

Thanks @mu88, it is very clear all up! Maybe you can add a point on how to install docker/moby on the Raspberry Pi.
Rest looks clear to me! And great you've added the links at the end as well.