Replies: 1 comment
-
While the container grants the privileges, the actual
User Mode Linux as in https://github.com/weber-software/diuid could be used to run BuildKit (or whatever else) without adding any extra security options, although its performance overhead is significant. Alternatively, VM-based runtimes such as Kata could be used to run BuildKit inside it, but nested virtualization is often missing in cloud, and it is not free from the performance overhead either. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
We are building a public free service for building packages and containers in our cloud. Anyone can upload a package or container description, and after some time receive the result.
The problem is that this service will definitely be attacked and people will try to hack it. We cannot trust what users upload as container descriptions. We cannot trust anyone, and the slightest security breach means that hackers will be able to break into other people's images and packages, and inject malicious code into them.
We recently completed a security audit and identified several container-related subprojects that all depend on BuildKit. And this is wonderful. It's good that an old, stable, well-maintained product with a history of many years is being used. Or so it would seem.
But the we discovered that the "rootless" image of BuildKit isn't truly rootless after all. (I'm very uncomfortable saying this because it sounds rude. I really want to be wrong about this issue. Please help me understand where I'm wrong!)
When I attempted to run BuildKit on our VM, I found that it requires several high-risk operations that could potentially compromise the entire cluster if any security vulnerability were exploited.
Regarding running rootless BuildKit within rootless Docker, I couldn't figure it out myself nor find any documentation on how to implement this setup.
If we run rootless BuildKit on standard Docker, we need to add the following configurations:
This allows non-root users to create new namespaces.
This removes restrictions on system calls inside the container, including mounting into /proc.
This option unconfines resources, including mounting capabilities.
This permits access to the host's proc filesystem instead of using an isolated one.
I tried to understand how to run this on Kubernetes and found this issue: #3897
As far as I understand the contents of the ticket, this grants BuildKit almost root-like privileges. In the linked issues, people even recommend running with --privileged, which would ruin the entire Kubernetes cluster's security.
All of these settings seem extremely dangerous. I'm genuinely concerned about implementing them.
We're now trying to rewrite all our BuildKit-specific code to use Podman, Buildah, or an alternative solution, but it's requiring an enormous amount of effort. Besides, BuildKit in general seems like a better solution in our case. Giving it up would be painful and sad.
Can you help us? How can we make this truly rootless and secure?
Beta Was this translation helpful? Give feedback.
All reactions