rootless podman driver

[ejiektpobehuk]

Steps to reproduce the issue:

1. minikube start --driver=podman

Full output of failed command:

😄  minikube v1.11.0 on Arch rolling
    ▪ KUBECONFIG=/path-to-my-config
✨  Using the podman (experimental) driver based on user configuration

❗  'podman' driver reported an issue: "sudo -k -n podman version --format {{.Version}}" exit status 1: sudo: a password is required
💡  Suggestion: Add your user to the 'sudoers' file: 'ejiek ALL=(ALL) NOPASSWD: /usr/bin/podman'
📘  Documentation: https://podman.io

💣  Failed to validate 'podman' driver

This means that with podman driver user is obligated to have root access without password wich is not minimal provilage.
It's possible to run podman rootless.

Here is a bit of Arch wiki on configuring podman not to require root permissions.

I suggest having a flag for this driver to either use sudo or not.

There is a good chance that rootless podman is not suitable for minikube yet due to it's shortcomings (this is a linkg to a particular version of podman)

[ejiektpobehuk]

might be a duplicate of #7480

[afbjorklund]

I suggest having a flag for this driver to either use sudo or not.

We require rootful podman (v1) at the moment. This is unlikely to change, same with dockerd --rootless.

If you are interested in trying to run kubernetes without root, you can look into the usernetes project ?

So this is more about #7963

You could also use a VM...

Currently the podman driver (and the docker driver) use privileged system containers, in order to emulate a "node."

Then they use crio-in-podman (or containerd-in-docker), for starting up the selected kubernetes container runtime.

[afbjorklund]

There is a good chance that rootless podman is not suitable for minikube yet due to it's shortcomings (this is a linkg to a particular version of podman)

A lot of these are blockers:

• No cgroup V1 Support
• No CNI Support

Others are just bad performance:

• fuse-overlayfs
• slirp4netns

There is no big use case for minikube supporting podman, and it becomes even smaller for rootless podman.

I think we need Cgroups v2 support to land in Kubernetes (and Docker), before we look at supporting it (v2)...

[ejiektpobehuk]

Sounds reasonable, thanks!

Should we keep this issue open for the bright future or you preferto create a new one once the time comes (if at all)?

[afbjorklund]

I think spinning up an entire kubernetes "node" inside a rootless container will be hard to accomplish, if at all...

We already see a lot of issues, from the difference between a virtual machine and a (rootful) system container.

So it would definitely be at the bottom of the list.

Minikube runs in virtualbox ("classic")
Minikube runs in "native" hypervisor, linux libvirt
Minikube runs in "native" hypervisor, win hyper-v
Minikube runs in "native" hypervisor, mac hyperkit

Minikube runs on docker engine, linux
Minikube runs on docker desktop, win
Minikube runs on docker desktop, mac
Minikube runs on sudo podman, linux

Already a lot of different platforms (8), to support.

And that's not even including the case when you run the installation right on the linux host with the none driver.

Which then runs into issues with different linux distributions, and with general lack of isolation (and multi-node!)

So I think it would be better off handled in usernetes.

There was some similar "root" discussions in #8257

[AkihiroSuda]

Work in progress for kind: kubernetes-sigs/kind#1727

Probably it can be easily ported to minikube as well

[AkihiroSuda]

think we need Cgroups v2 support to land in Kubernetes (and Docker)

Already landed on their master branches (Kubernetes 1.19, Docker 20.0X)

[afbjorklund]

think we need Cgroups v2 support to land in Kubernetes (and Docker)

Already landed on their master branches (Kubernetes 1.19, Docker 20.0X)

Good to hear! Then we just need those to be released. I guess Podman 2.0.x is already out

[medyagh]

I look forward to see a prototype of rootless docker/podman in minikube

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[AkihiroSuda]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[AkihiroSuda]

/remove-lifecycle stale

[AkihiroSuda]

kind works with Rootless Podman as well as Rootless Docker now: kubernetes-sigs/kind#1935

Should be easily portable to minikube.

[afbjorklund]

Guess we just have to add some checks, for latest Podman/Docker (with cgroups v2) and latest Kubernetes (>= 1.20)

[AkihiroSuda]

The first step toward this is to bring proper support for cgroup2: #11310

[k8s-triage-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[AkihiroSuda]

/remove-lifecycle stale

[AkihiroSuda]

I opened PR #12359 for rootless docker, but this PR doesn't cover rootless Podman yet, because I'm not sure how the minikube CLI should work for rootless Podman.

The current minikube start --driver=podman launches rootful podman with sudo, and I assume we don't want to break the current behavior.

Do we need additional flags like --driver=podman --podman-rootless=true, or do we need a new driver like --driver=podman-rootless ?

[afbjorklund]

One could sniff at the $CONTAINER_HOST, and only do the sudo if not having access to the unix socket or something.

The problem is that half of the podman commands run directly, and half of them (like through bindings) run remotely.

Instead of all of them going through the remoting layer, like is the case with docker (there is always a server daemon)

Do we need additional flags like --driver=podman --podman-rootless=true, or do we need a new driver like --driver=podman-rootless ?

In the end, having rootless as a top-level boolean is probably the best. To make it clear to everything, what is the setup ?

Having two separate drivers seems like a lot of code duplication, especially when it is not needed for the docker driver.

We could have it as a boolean for docker too, but I guess you just ask the daemon about the current state ? (info)

[AkihiroSuda]

PR: #12901

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[AkihiroSuda]

/remove-lifecycle stale

PR #12901 has received two LGTMs, but not merged yet