Version 4.x not updating with apt update Ubuntu 20.04

[TCROC]

On Ubuntu 20.04, Podman is stuck at version 3.4.2. Running apt update doesn't update it to the 4.x version. I followed the instructions here: https://podman.io/getting-started/installation.html#linuxmint-20x.

[afbjorklund]

Those packages (from Kubic) are not updated anymore, the instructions should be removed (for Mint, like they were for Ubuntu)

containers/podman.io@4c7bd52

[mythi]

Those packages (from Kubic) are not updated anymore, the instructions should be removed (for Mint, like they were for Ubuntu)

has there been any announcements about kubic? we've installed buildah from it and that disappeared completely.

[TCROC]

Those packages (from Kubic) are not updated anymore, the instructions should be removed (for Mint, like they were for Ubuntu)

What is the new way to install Podman on Ubuntu 20.04?

[afbjorklund]

Here was the update:
https://podman.io/blogs/2022/04/05/ubuntu-2204-lts-kubic.html

[TCROC]

Ok I see. So Podman 4.x is available on Ubuntu 22.04 if I upgrade?

[afbjorklund]

It is available on Fedora and RHEL, not on Ubuntu (yet)

https://packages.ubuntu.com/search?keywords=podman&searchon=names&exact=1&suite=all&section=all

[TCROC]

Any plans for when it will land on Ubuntu? 👀 :)

[afbjorklund]

https://packages.debian.org/search?keywords=podman&searchon=names&exact=1&suite=all&section=all

Not in "Sid" yet, see https://wiki.ubuntu.com/Upstream

[TCROC]

Ah ok cool. And am I able to add that as a PPA on Ubuntu?

[afbjorklund]

I believe you would need to build it from the source code.

• When will you release v4 binaries (ie v4.0.2) ? #13407 (comment)

The same goes for all the required dependencies, as well...

• [kicbase] Proposal: install podman from binary tgz, not from deb kubernetes/minikube#14198

---

More discussion:

• Ubuntu 22 apt-get install podman is only version 3.4.4 #14065
• Wsl Ubuntu 20.04 - Upgrade Podman version 3.4.2--> v4.1.0 #14260
• No updates on Kubic project for podman 3.4.x? #12265

[TCROC]

What is the reasoning for not including 4.x in an Ubuntu ppa?

[lsm5]

@TCROC Ubuntu ppas are a huge PITA (at least for me (Fedora guy)). Also, no debian support from what I can tell.

RE: debian experimental repo that @afbjorklund mentioned above, you should be able to install the deb package on ubuntu, at least that's what the debian maintainer told us. I don't know if you'd be able to add it as a repo source to Ubuntu though.

Also, I have a WIP kubic repo at https://build.opensuse.org/project/show/devel:kubic:libcontainers:unstable . This will likely not be officially documented for quite some time as I cannot promise long term maintainability / support yet. But if you're willing to try it out, be my guest. This repo currently allows you to use podman v4.1.0 on debian sid and ubuntu 22.04 LTS along with the latest network stack (netavark and aardvark-dns).

Closing..

[wuestkamp]

Is there still a way to install Podman (any version) on Ubuntu 20.04?
Because this throws gpg: no valid OpenPGP data found:

. /etc/os-release
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/testing/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:testing.list
curl -L "https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/testing/xUbuntu_${VERSION_ID}/Release.key" | sudo apt-key add -
sudo apt-get update -qq
sudo apt-get -qq -y install podman

[TCROC]

@wuestkamp I suggested something over here: #14332

Sounds like Snap or Flatpak might be a possibility 👀. Still waiting on an official comment.

[afbjorklund]

Is there still a way to install Podman (any version) on Ubuntu 20.04?

The kubic packages are still the "best", but they do have security issues.

EDIT: Nope

[lsm5]

Given all the love for podman and ubuntu 20.04, I have restored podman on the devel:kubic:libcontainers:stable project. Again, this is the same v3.4.2 build that used to exist prior and is vulnerable to the CVEs mentioned above. So, use at your own risk.

Builds should be published in a few mins.

[crfrolik]

I'm confused by this. Why is it still so difficult to get 4.X installed on Ubuntu?

[vrothberg]

I'm confused by this. Why is it still so difficult to get 4.X installed on Ubuntu?

It's an immense amount of work. It is not "only" packaging podman but many of its runtime dependencies including config files, man pages etc. Plus maintaining all of that.

Since podman is now in the main repositories of Ubuntu, we prefer users to use these packages.

[alexanderadam]

Since podman is now in the main repositories of Ubuntu, we prefer users to use these packages.

The current Ubuntu 22.04 LTS will be supported for 10 years.
So you suggest that its users are stuck with Podman 3 for ten years?

I also asked on StackOverflow for a solution but so far no one got an answer how to get Podman 4 in a sane manner that doesn't involve setting up and maintaining binaries manually.

It's probably pointless to mention again that an official PPA would solve that for everybody. 😉

…but I also had some issues with Kernel built-in overlayfs so there's that. 🤷

[vrothberg]

If people in the community step up to do this work, they are more than welcome.

The Podman maintainers simply don't have the capacity to ship/package new versions to all distributions since the support matrix is just too big.

…but I also had some issues with Kernel built-in overlayfs so there's that.

Please open issues on GitHub. Twitter is not a good platform to report bugs or issues; it's very easy that Tweets get lost in the noise.

It's probably pointless to mention again that an official PPA would solve that for everybody.

As much as I sympathize with your desire, there is nothing we can do. As mentioned above, people are more than welcome to step up and create a community PPA. But realize that it is a lot of work.

In my view and experience, it's best to work directly together with the distributions. Did you reach out to the Ubuntu community/Canonical and asked whether/how Podman can be updated for LTS versions?

[afbjorklund]

The LTS versions (just like EL) are supposed to ship with old versions, that is sort of their selling point ?

So normally you would have to use something like "backports" (EPEL), to get the latest and greatest...

Another possibility is doing a 3.5 maintenance release.

Or at least getting the deb version updated, from 3.4.2.

[vrothberg]

The LTS versions (just like EL) are supposed to ship with old versions, that is sort of their selling point ?

I think that depends on the individual needs. This is also why there are streams in RHEL for shipping different versions of the container tools.

AFAIK, Ubuntu updates the kernel for LTS versions at some point as well. The balance of "stability" and "new features" is delicate and maintenance for such a long time is a herculean task.

[afbjorklund]

I guess that is now up to the Ubuntu maintainers, how they want to handle the "podman" package... Stay at 3.4 or go to 4.x

Still somewhat ironic that it is easier to upgrade Podman on Mac and on Win, but I guess you can use machine on Linux too.

[lsm5]

It's probably pointless to mention again that an official PPA would solve that for everybody. wink

@alexanderadam I have an UNSTABLE Kubic repo at https://build.opensuse.org/project/show/devel:kubic:libcontainers:unstable . For the near future, it won't get any more official than that.

This repo gets the latest upstream podman and other packages soon after upstream release. I did try it myself sometime ago and AFAICT, 22.04 should mostly just work.

Not recommended for production use, so please be careful where you use it. Let me know what your experience is like.

UPDATE: I'm checking with github actions people about using this repo in their environments and will also update the podman installation docs once I'm certain this works reasonably well.

[Korijn]

@lsm5 Let me just get out of the way that I completely understand your motivations and reasons not to provide a stable PPA. It's a big piece of work and time is a limited resource.

I just want to make sure you are aware that this is also blocking public CI environments from adoption of podman v4. Github actions, azure pipelines, etc are all also stuck.

[matt-deboer]

Thanks much, @lsm5 ! successful build/install now on x86_64 22.04

[alexanderadam]

Is there any chance also to have the stable versions available somehow or is the integration effort difference between unstable and too huge?

[lsm5]

Is there any chance also to have the stable versions available somehow or is the integration effort difference between unstable and too huge?

@alexanderadam The repo is called (and will continue to be called) unstable mainly so people are aware it's not recommended for production use. Only the stable upstream releases are being sent to this unstable repo but the packages don't go through debian's or ubuntu's official QE process. I stopped sending RC builds to this repo recently. There might be scope for some automated QE but it's not too high on my priority list atm. I'll gladly take any help from the community toward more testing.

Does that answer your Q or did you mean something else?

[alexanderadam]

the packages don't go through debian's or ubuntu's official QE process

I'm fully aware of that. This is also true for other repos for other projects (even stable) ones (i.e. dl.google.com/linux/chrome/deb/ for Chrome or libreoffice/ppa for LibreOffice).

My question was rather whether it would be possible to make stable versions available via a repo as well.

Does that answer your Q or did you mean something else?

I guess it does. If I understood it correctly, it would be possible to provide a stable channel in general but its priority is low and hence it will currently stay that way that there's production ready solution for Podman versions higher 3.x.

Hence thank you for your explanation and your work.

[lsm5]

I guess it does. If I understood it correctly, it would be possible to provide a stable channel in general but its priority is low and hence it will currently stay that way that there's production ready solution for Podman versions higher 3.x.

We did provide stable and testing channels back when I was building deb packages the rather conventional way. Those are not being updated anymore.

RE: production ready, I don't think the podman team will ever be comfy recommending the OBS / kubic packages for production use. Having a stable channel would, in my opinion, raise expectations for the deb packages, which won't be in scope for the podman team, at least for the foreseeable future. Even if we have all tests pass with the kubic packages, the podman installation docs will still only recommend the official distro packages for production use.

So, I guess we are pretty much on the same page. I'm just trying to keep expectations super low 😄

[jeremyn]

There's a known problem where VS Code's remote development functionality is a little glitchy with Podman v3 but works with v4, see microsoft/vscode-remote-release#7175. I prefer Ubuntu so I'm interested in this issue.

Obviously, keeping expectations low, and I don't mean to pressure anyone. However I'm unclear if anyone really owns this problem and what will happen by default. Have either Red Hat or Canonical shown any signs of stepping up to move Podman to v4 by, say, Ubuntu 24.04? If this all really depends on a couple unpaid heroes keeping it updated in their free time, then, no offense intended, but that's basically the same as saying that it's unsupported and people should either move to a RHEL-based distro if they want to keep using Podman, or to Docker on Ubuntu.

Is this pretty much how things are? Thoughts?

[afbjorklund]

Have either Red Hat or Canonical shown any signs of stepping up to move Podman to v4 by, say, Ubuntu 24.04?

I think you need to ask the unsung heroes, like @siretart, about the roadmap for the packaging in debian:

https://packages.debian.org/search?keywords=podman&searchon=names&exact=1&suite=all&section=all

Once it is included in debian, that increases the chances of the package being selected for future ubuntu...

https://packages.ubuntu.com/search?keywords=podman&searchon=names&exact=1&suite=all&section=all

[jeremyn]

Not to step on any toes but as I understand it, Podman is essentially a Red Hat project even if it's open source and some work comes from outside Red Hat. So Red Hat's intentions regarding the product are important even if specific people like @siretart have so far taken it on themselves to do extra work. Fedora Rawhide already has Podman v4.3.0 which was released less than three weeks ago. So we can say Fedora and its downstreams are a higher-priority target for Red Hat than Debian and its downstreams, and maybe Fedora etc is the exclusive target.

In any case my interest is fixing the VS Code problem I linked in my previous comment by getting Podman v4 in Ubuntu. The VS Code team (let's just call them "Microsoft") hasn't really moved forward on the issue other than noting that it's fixed in v4. I'm trying to find out whether any corporate interest has committed to ever getting Podman v4 into Debian (or Ubuntu) even at a slower pace than Fedora. Otherwise it comes down to Microsoft indirectly putting their problem onto @siretart's shoulders, which is not something they should count on and which I can point out to them in the other issue.

[hkrutzer]

I think it has been made quite clear that Red Hat will not support Debian or Ubuntu:

The Podman maintainers simply don't have the capacity to ship/package new versions to all distributions since the support matrix is just too big.

as well as from the fact that this issue is closed.

One can reach out to Ubuntu as was suggested in the same comment:

In my view and experience, it's best to work directly together with the distributions. Did you reach out to the Ubuntu community/Canonical and asked whether/how Podman can be updated for LTS versions?

or for example sponsor someone to create the package.

[mheon]

@lsm5 is on the Podman team and continues to put a lot of time into packaging Podman for Ubuntu, but only via OBS; all efforts regarding the official repos are handled by @siretart and the Ubuntu maintainers.

[vrothberg]

I think it has been #14302 (comment) that Red Hat will not support Debian or Ubuntu:

I want to clarify just to make sure the text isn't misinterpreted without reading the linked comment. I also want to avoid that we generalize too much.

"Support" can mean many things. As an upstream project, we support all distributions equally. Bugs will be fixed and features will be implemented regardless of the distribution.

What this issue is about is package maintenance. There is an overlap of upstream contributors and downstream packagers but it is practically impossible for upstream to maintain packages for all distributions. Some users want it stable, some users want it rolling, etc. and all have different life and release cycles.

Many upstream contributors are Red Hat employees, so Fedora is really quick in consuming new versions. But ArchLinux, OpenSUSE and others are just as fast.

So what people in this thread are asking for is accelerating downstream packaging for Debian and Ubuntu. Doing it faster and providing stable and fast/rolling versions of the packages is a lot of work that will exceed volunteers' and upstream capabilities.

As mentioned above, I recommend reaching out to the Debian/Ubuntu community and help them. Or open a ticket at Ubuntu/Canonical. Really, please talk to them so they realize a business need to provide Podman support for their products.

[afbjorklund]

And it would probably still be a good idea to update to podman 3.4.7, even if it is not supported anymore either.

docker@minikube:~$ podman --version
podman version 3.4.2

On Ubuntu 20.04, Podman is stuck at version 3.4.2. Running apt update doesn't update it to the 4.x version

[jeremyn]

I came here for information on Podman updates in Ubuntu so I could provide that info in the VS Code issue. I have that information now, so, thanks.

That said, I'll add that I don't think any of the external users are talking about a support matrix for all distributions, only polishing and officially supporting the build that currently exists in the Kubic repository, for Ubuntu LTS, which is very popular and has a predictable and slow release cycle. We don't need to put Arch etc into the mix. Also Podman is a competitive advantage for Red Hat and I don't think it's totally reasonable to ask unaffiliated users to lobby Canonical to improve support for a competitor's product. If Red Hat wants that to happen, they can contact Canonical first, then users can show support.

None of this is to attack any of the Red Hat employees participating in this discussion. It's just that there are different perspectives from the inside looking out, "we are too busy to support Ubuntu", and the outside looking in, "if Red Hat wanted Ubuntu support, they could pay for it," and both of these perspectives can be true at the same time.

[jeremyn]

In the VS Code issue I linked, @dawidd6 had the excellent recommendation of using Homebrew to install updated Podman on Linux. This seems to check out since:

1. The Podman documentation recommends using Homebrew for Mac in the installation instructions.
2. The formula page for Podman says Linux is supported.

Is there any interest in updating the Podman installation instructions to point to Homebrew on Linux for distributions with outdated versions of Podman? I can make a separate issue to discuss that, if that would help.

[lsm5]

@jeremyn sure, that's best done on https://github.com/containers/podman.io

[TCROC]

@jeremyn YES! This got me the up to date podman! Should def be the recommended way for Ubuntu. Here are the steps:

sudo apt install build-essentials
sudo apt install uidmap

Install homebrew: https://brew.sh/

brew install podman

ez pz! :)

[dawidd6]

Good to know it's working for others too :)

[jeremyn]

I tried Podman with Homebrew on Linux and can also confirm it seems fine.

Someone else can take the lead on pushing forward a documentation issue, though. Personally I'll probably just put up with the glitchy behavior I have with VS Code using Podman from the official repositories and wait for an update, or switch to Docker or to Fedora or otherwise change things around.

[siretart]

Thank you for all the support and kind words on distro packaging.

Maybe a quick status update, I've uploaded podman 4.3.1 to debian/experimental yesterday. I'd really appreciate if people could give those packages a shot and file bugs using reportbug, the debian bug submission tool. Bonus points if you could help me with the open bugs at https://bugs.debian.org/src:libpod -- just sent a followup at NNNN@bugs.debian.org (with NNNN being the bug number) whether you can reproduce it or not.

I plan on uploading 4.3.1 to unstable later this week. This will give the packages much more attention, and I expect more bugs to surface.

As for ubuntu, the packages are mostly in sync with debian/unstable, but ubuntu packages runc and docker differently, which causes an unfortunate delta that needs to be handled with manually. Nothing unsurmountable, just annoying.

I keep getting asked about backports to bullseye (and by extension, to released versions of ubuntu). The short answer is that I don't have capacity for this. It is not "just" about podman, it is the whole container stack: containers/storage, containers/image, containers/common, containers/buildah and containers/podman, all of which need to be backported in that order. Also, containers/storage has a number of system-related dependencies that may or may not need to be backported as well. It is a less than trivial backport.

[TCROC]

@jeremyn YES! This got me the up to date podman! Should def be the recommended way for Ubuntu. Here are the steps:

sudo apt install build-essentials
sudo apt install uidmap

Install homebrew: https://brew.sh/

brew install podman

ez pz! :)

Update regarding using homebrew. I found that homebrew actually installed a bunch of dependencies that were already installed on my Ubuntu distro causing conflicts. It would be nice if it detected software already installed on my pc. Would this be more appropriate as a podman or homebrew issue?

[jeremyn]

@TCROC Almost certainly Homebrew.

[rhatdan]

Yes this has nothing to do with the upstream podman effort.

[uogbuji]

This is a complete dead end for me. Trying to use VS Code & Running into

Error: unable to connect to Podman socket: server API version is too old. Client "4.0.0" server "3.4.4"

No I'm not installing homebrew on my Linux Mint box, just to get a year-old version of Podman. I guess it's back to Docker.

[hkrutzer]

https://launchpad.net/ubuntu/+source/libpod

[uogbuji]

So I guess until Linux Mint is updated to Lunar Lobster, or the Kubic PPA is supported again, I'd have to either build podman from scratch or use homebrew. sigh Look, I'm not meaning to be difficult, but I'm just trying to get work done and have already burned a couple of hours on this. I don't really have a choice but to just use Docker, which I already know will work with VS Code remote with no fiddling. Respect to you, though, @hkrutzer, for replying to my peeved note.

[lsm5]

So I guess until Linux Mint is updated to Lunar Lobster, or the Kubic PPA is supported again, I'd have to either build podman from scratch or use homebrew. sigh Look, I'm not meaning to be difficult, but I'm just trying to get work done and have already burned a couple of hours on this. I don't really have a choice but to just use Docker, which I already know will work with VS Code remote with no fiddling. Respect to you, though, @hkrutzer, for replying to my peeved note.

the kubic unstable repo does have the latest v4.5.1 https://build.opensuse.org/package/show/devel:kubic:libcontainers:unstable/podman . But beware, use in production at your own risk. That's the extent of kubic support the podman team will be able to provide for the foreseeable future.