Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API for "podman" (service), similar to (docker) "daemon" #998

Closed
afbjorklund opened this issue Apr 26, 2021 · 19 comments
Closed

Add API for "podman" (service), similar to (docker) "daemon" #998

afbjorklund opened this issue Apr 26, 2021 · 19 comments

Comments

@afbjorklund
Copy link
Contributor

afbjorklund commented Apr 26, 2021

Currently we have daemon.Image and daemon.Write

We would also need podman.Image and podman.Write.

kubernetes/minikube#11063

pkg/minikube/image/podman.go

It does not have any daemon, when not running remotely...

Current root authentication is handled by running with sudo.

docker podman
docker save sudo -n podman save
docker tag sudo -n podman tag
docker load sudo -n podman load
@imjasonh
Copy link
Collaborator

+1 to having support for this. Is there a Go client that podman load and podman save underneath, or is execing the CLI really the best option we have?

In general my advice would be to iterate on the podman package in the minikube codebase or wherever else you want until it meets your needs, then we can figure out whether it belongs in this module or outside. If we move it into this codebase it could help us bring Podman support to tools built on it, like buildpacks and ko.

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Apr 26, 2021

+1 to having support for this. Is there a Go client that podman load and podman save underneath, or is execing the CLI really the best option we have?

There has been various attempts to use "libpod" and the API, but it keeps breaking and has some other issues...

here was one example: https://github.com/hashicorp/nomad-driver-podman

I think they settled on using the remote API... Haven't checked with later Podman, if the situation has improved ?

some of the issues found: containers/podman#3445 | hashicorp/nomad#5312


Note that each version of Podman has it's own API.

Also there is no way to authenticate, have to run as root.

export CONTAINER_HOST=unix:/run/podman/podman.sock

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Apr 26, 2021

Upstream suggests that using the remote API is the best bet for a library, and that UNIX group is the best bet for authentication.

So that pushes the burden of configuring and starting the unit to the user, to make sure that podman.socketunit is available:

    groupadd -f -r podman

    #systemctl edit podman.socket
    mkdir -p /etc/systemd/system/podman.socket.d
    cat >/etc/systemd/system/podman.socket.d/override.conf <<EOF
[Socket]
SocketMode=0660
SocketUser=root
SocketGroup=podman
EOF
    systemctl daemon-reload
    echo "d /run/podman 0770 root podman" > /etc/tmpfiles.d/podman.conf
    systemd-tmpfiles --create

    systemctl enable podman.socket
    systemctl start podman.socket

But if they do want to do that, then pkg/v1/daemon can be used : just set DOCKER_HOST=unix:///run/podman/podman.sock

I still think sudoers is easier to explain, and seems weird to be running a daemon for a "daemon-less" container runtime ?

@jonjohnsonjr
Copy link
Collaborator

I weakly prefer doing something boring/gross like execing the client just because bringing in the docker client for daemon did so much damage to the dependency graph.

@afbjorklund
Copy link
Contributor Author

afbjorklund commented May 17, 2021

In theory (as in: maybe for these three commands), the same wrapper would also work for using the docker CLI client.

If you want to get rid of "github.com/docker/docker" too, from "github.com/google/go-containerregistry/pkg/v1/daemon"

@github-actions
Copy link

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.

@jonjohnsonjr
Copy link
Collaborator

the same wrapper would also work for using the docker CLI client

Yeah I could see this being nice, but I'm a little worried about the use case where you don't have docker installed locally and want to interact with a remote daemon. We can't shell out to docker :/

I do think it would be interesting to define some kind of "plugin" interface where go-containerregistry could shell out to things, which would allow you to extend e.g. crane to other targets without blowing up the dependency graph, but I'm not sure what that interface would look like.

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Aug 16, 2021

For now I think we will settle for having the podman users set up a docker socket, since e.g. Skaffold also hard-codes docker client.

local
export DOCKER_HOST=unix:///run/podman/podman.sock

remote
sudo ln -sf /run/podman/podman.sock /var/run/docker.sock

If wanting to support podman "properly", I think libpod would be the "sane" way. Should be stable for 3.x, but might break in 4.0

There are some concerns about the amount of bloat that the go client pulls in....

containers/podman#6866

libpod-build-size-treemap

@afbjorklund
Copy link
Contributor Author

bringing in the docker client for daemon did so much damage to the dependency graph.

note that importing the podman client adds a dependency on github.com/docker/docker

@afbjorklund
Copy link
Contributor Author

The natural implementation of this would talk directly to podman (libpod), but then there is the authentication issue...
So it seems it will only be possible for rootless podman, which isn't of any use to our use case (running minikube)

I'm inclined to close this ticket, and leave it at that go-containerregistry only supports Docker and Docker sockets.
We could do a prototype of a rootless implementation for podman (with Go API, not CLI), if there was any interest.

Will leave the current CLI workarounds in Minikube, for the small amount of Podman users (everybody uses Docker)

Maybe revisit witth a new issue a rainy day (or when Podman 4 is out), but don't care to maintain any long-living PRs...

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Aug 21, 2021

For the minikube image command, --daemon will mean Docker daemon and --remote will mean a Docker registry.
Everyone else will have to speak their protocol. Locally (in the cache directory), we will use the Docker tarball format.

See kubernetes/minikube#12270

Maybe this OCI thing will catch on eventually, i.e. any year now.

(containerd uses OCI archives, but includes a Docker manifest)

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Aug 22, 2021

The pkg/v1/podman implementation was quite straight-forward (except for some quirks with Save and LoadOptions in API)

https://github.com/afbjorklund/go-containerregistry/blob/podman/pkg/v1/podman/client.go

But as expected, the vendored dependencies are not so fun: 45 files changed, 1461 insertions(+), 249 deletions(-)

	github.com/containers/podman/v3 v3.3.0
github.com/containers/podman/v3@v3.3.0 k8s.io/apimachinery@v0.21.3
github.com/containers/podman/v3@v3.3.0 k8s.io/api@v0.21.3
github.com/containers/podman/v3@v3.3.0 gopkg.in/yaml.v3@v3.0.0-20210107192922-496545a6307b
github.com/containers/podman/v3@v3.3.0 golang.org/x/sys@v0.0.0-20210616094352-59db8d763f22
github.com/containers/podman/v3@v3.3.0 golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c
github.com/containers/podman/v3@v3.3.0 golang.org/x/crypto@v0.0.0-20210322153248-0c34fe9e7dc2
github.com/containers/podman/v3@v3.3.0 go.etcd.io/bbolt@v1.3.6
github.com/containers/podman/v3@v3.3.0 github.com/vishvananda/netlink@v1.1.1-0.20201029203352-d40f9887b852
github.com/containers/podman/v3@v3.3.0 github.com/vbauerster/mpb/v6@v6.0.4
github.com/containers/podman/v3@v3.3.0 github.com/uber/jaeger-client-go@v2.29.1+incompatible
github.com/containers/podman/v3@v3.3.0 github.com/syndtr/gocapability@v0.0.0-20200815063812-42c35b437635
github.com/containers/podman/v3@v3.3.0 github.com/stretchr/testify@v1.7.0
github.com/containers/podman/v3@v3.3.0 github.com/spf13/pflag@v1.0.5
github.com/containers/podman/v3@v3.3.0 github.com/spf13/cobra@v1.2.1
github.com/containers/podman/v3@v3.3.0 github.com/sirupsen/logrus@v1.8.1
github.com/containers/podman/v3@v3.3.0 github.com/rootless-containers/rootlesskit@v0.14.4
github.com/containers/podman/v3@v3.3.0 github.com/pmezard/go-difflib@v1.0.0
github.com/containers/podman/v3@v3.3.0 github.com/pkg/errors@v0.9.1
github.com/containers/podman/v3@v3.3.0 github.com/opencontainers/selinux@v1.8.4
github.com/containers/podman/v3@v3.3.0 github.com/opencontainers/runtime-tools@v0.9.0
github.com/containers/podman/v3@v3.3.0 github.com/opencontainers/runtime-spec@v1.0.3-0.20210326190908-1c3f411f0417
github.com/containers/podman/v3@v3.3.0 github.com/opencontainers/runc@v1.0.1
github.com/containers/podman/v3@v3.3.0 github.com/opencontainers/image-spec@v1.0.2-0.20190823105129-775207bd45b6
github.com/containers/podman/v3@v3.3.0 github.com/opencontainers/go-digest@v1.0.0
github.com/containers/podman/v3@v3.3.0 github.com/onsi/gomega@v1.14.0
github.com/containers/podman/v3@v3.3.0 github.com/onsi/ginkgo@v1.16.4
github.com/containers/podman/v3@v3.3.0 github.com/mrunalp/fileutils@v0.5.0
github.com/containers/podman/v3@v3.3.0 github.com/moby/term@v0.0.0-20201216013528-df9cb8a40635
github.com/containers/podman/v3@v3.3.0 github.com/mattn/go-colorable@v0.1.8
github.com/containers/podman/v3@v3.3.0 github.com/json-iterator/go@v1.1.11
github.com/containers/podman/v3@v3.3.0 github.com/hpcloud/tail@v1.0.0
github.com/containers/podman/v3@v3.3.0 github.com/hashicorp/go-multierror@v1.1.1
github.com/containers/podman/v3@v3.3.0 github.com/gorilla/schema@v1.2.0
github.com/containers/podman/v3@v3.3.0 github.com/gorilla/mux@v1.8.0
github.com/containers/podman/v3@v3.3.0 github.com/google/uuid@v1.3.0
github.com/containers/podman/v3@v3.3.0 github.com/google/shlex@v0.0.0-20181106134648-c34317bd91bf
github.com/containers/podman/v3@v3.3.0 github.com/godbus/dbus/v5@v5.0.4
github.com/containers/podman/v3@v3.3.0 github.com/ghodss/yaml@v1.0.0
github.com/containers/podman/v3@v3.3.0 github.com/fsnotify/fsnotify@v1.4.9
github.com/containers/podman/v3@v3.3.0 github.com/docker/go-units@v0.4.0
github.com/containers/podman/v3@v3.3.0 github.com/docker/go-plugins-helpers@v0.0.0-20200102110956-c9a8a2d92ccc
github.com/containers/podman/v3@v3.3.0 github.com/docker/go-connections@v0.4.0
github.com/containers/podman/v3@v3.3.0 github.com/docker/docker@v20.10.7+incompatible
github.com/containers/podman/v3@v3.3.0 github.com/docker/distribution@v2.7.1+incompatible
github.com/containers/podman/v3@v3.3.0 github.com/digitalocean/go-qemu@v0.0.0-20210209191958-152a1535e49f
github.com/containers/podman/v3@v3.3.0 github.com/davecgh/go-spew@v1.1.1
github.com/containers/podman/v3@v3.3.0 github.com/cyphar/filepath-securejoin@v0.2.3
github.com/containers/podman/v3@v3.3.0 github.com/cri-o/ocicni@v0.2.1-0.20210621164014-d0acc7862283
github.com/containers/podman/v3@v3.3.0 github.com/coreos/stream-metadata-go@v0.0.0-20210225230131-70edb9eb47b3
github.com/containers/podman/v3@v3.3.0 github.com/coreos/go-systemd/v22@v22.3.2
github.com/containers/podman/v3@v3.3.0 github.com/containers/storage@v1.34.1
github.com/containers/podman/v3@v3.3.0 github.com/containers/psgo@v1.5.2
github.com/containers/podman/v3@v3.3.0 github.com/containers/ocicrypt@v1.1.2
github.com/containers/podman/v3@v3.3.0 github.com/containers/image/v5@v5.15.2
github.com/containers/podman/v3@v3.3.0 github.com/containers/conmon@v2.0.20+incompatible
github.com/containers/podman/v3@v3.3.0 github.com/containers/common@v0.42.1
github.com/containers/podman/v3@v3.3.0 github.com/containers/buildah@v1.22.3
github.com/containers/podman/v3@v3.3.0 github.com/containernetworking/plugins@v0.9.1
github.com/containers/podman/v3@v3.3.0 github.com/containernetworking/cni@v0.8.1
github.com/containers/podman/v3@v3.3.0 github.com/container-orchestrated-devices/container-device-interface@v0.0.0-20210325223243-f99e8b6c10b9
github.com/containers/podman/v3@v3.3.0 github.com/checkpoint-restore/go-criu/v5@v5.1.0
github.com/containers/podman/v3@v3.3.0 github.com/checkpoint-restore/checkpointctl@v0.0.0-20210301084134-a2024f5584e7
github.com/containers/podman/v3@v3.3.0 github.com/buger/goterm@v0.0.0-20181115115552-c206103e1f37
github.com/containers/podman/v3@v3.3.0 github.com/blang/semver@v3.5.1+incompatible
github.com/containers/podman/v3@v3.3.0 github.com/BurntSushi/toml@v0.4.1

@afbjorklund afbjorklund changed the title Add API for (rootful) "podman", similar to (docker) "daemon" Add API for "podman" (service), similar to (docker) "daemon" Aug 22, 2021
@imjasonh
Copy link
Collaborator

I weakly prefer doing something boring/gross like execing the client just because bringing in the docker client for daemon did so much damage to the dependency graph.

If we added a pkg/v1/podman today I think we'd definitely put it in its own Go module. I think we'd do the same for pkg/v1/daemon if we were starting over today (maybe even a separate repo for both). If we add a podman module, I'd probably want to move daemon out too while we're at it.

If there's still interest in getting pkg/v1/podman into this repo, I'd still be interested in reviewing it.

@imjasonh imjasonh reopened this Jul 20, 2022
@afbjorklund
Copy link
Contributor Author

What do you plan to do for containerd then ?

@jwcesign
Copy link

jwcesign commented Jul 20, 2022

So for now, podman-support code is not merged into main because:

  • need to run as root with podman system service -t 5000
  • give normal user right to access /run/podman/podman.sock

Is there any other reason?

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Jul 20, 2022

You don't need to be root to access the user socket. Normally systemd starts the service for you, when the Unix socket is accessed.

It is only if you want to give a certain group of users root access through the socket, that you need special setup (like with docker)

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Jul 20, 2022

Note that a default user does not have access to Docker socket either, unless they are added to that root-equivalent group.

https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user

And that you can run docker as rootless too, and change the DOCKER_HOST to match (or something similar in api)

https://docs.docker.com/engine/security/rootless/#install

@afbjorklund
Copy link
Contributor Author

afbjorklund commented Sep 17, 2022

It seems like this would have to use podman.sock and nerdctl.sock, instead of talking to a docker.sock

DOCKER_HOST=unix:///run/podman/podman.sock

DOCKER_HOST=unix:///var/run/nerdctl.sock

Because of how hard the "daemon" is coded into the API, it seems unlikely that it will ever disappear...

sudo podman

sudo nerdctl

@github-actions
Copy link

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.

@afbjorklund afbjorklund closed this as not planned Won't fix, can't repro, duplicate, stale Dec 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants