Kind cluster not visible with Podman 5 rc7

[odockal]

Bug description

I do not see kind cluster in podman destkop. No error was shown in the console log regarding kind cluster creation. I am using podman 5 (rc7) with hyperv provider. The Machine is rootfull with user mode networking enabled.

Operating system

Windows 10

Installation Method

Installer from website/GitHub releases

Version

1.8.0

Steps to reproduce

Install podman desktop and podman 5 rc 7 from installer.

1. Create new podman machine - rootful and with usermode networking enabled
2. Create a kind cluster
3. Kind cluster created
4. assert: Podman Desktop sees the kind related resources (images, containers, etc.)
   actual result: Podman desktop does not show the cluster resources nor container objects.

Relevant log output

VM37:2 main ↪️ [podman]  Can't get machine usermode resource usage error Error: no running provider for the matching container
console.<computed>	@	VM37:2
(anonymous)	@	VM37:2
emit	@	VM36 sandbox_bundle:2
onMessage	@	VM36 sandbox_bundle:2

Additional context

I am observing that podman cli works nice, If I output podman images I can see kindest image present.

[jeffmaury]

Does it work with podman CLI ?

[odockal]

I think that the problem is in podman desktop not being using right connection to the socket, or socket has some problems.
On cli everything seems fine, image/container is running, kind cluster is available.

[cbr7]

I just encountered this issue with both kind and minikube clusters while testing PD 1.9.0 with podman 5.0.1 installed using applehv virtualization.

[benoitf]

on my end I see the kind cluster

and container is running

[benoitf]

and I'm using podman v5

Client:       Podman Engine
Version:      5.0.1
API Version:  5.0.1
Go Version:   go1.22.1
Git Commit:   946d055df324e4ed6c1e806b561af4740db4fea9
Built:        Mon Apr  1 16:05:47 2024
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      5.0.1
API Version:  5.0.1
Go Version:   go1.21.8
Built:        Mon Apr  1 02:00:00 2024
OS/Arch:      linux/arm64

[benoitf]

so do you see the kind container in container list, what are the logs of the kind cluster creation, etc ?

[cbr7]

@benoitf no, yesterday when I encountered this issue the containers were not visible in the container list in PD nor were the images visible in the Images page from PD, however they were visible when checked with podman CLI. Both clusters were created correctly and were operational when checked in kind and minikube CLI.

[benoitf]

if there is a desynchro between CLI and Desktop

report the podman system connection list value, podman machine inspect value

then in Podman Desktop:
report if there are log failures
do you see 'event' when you're for example starting or stopping a container using the CLI

goes to the light bulb and stores
report the number of containers before and after trying a manual refresh

Click on 'reconnect' as well to see if it's because connection has been dropped

[cbr7]

@benoitf currently I'm testing the migration as such had to remove the podman version that was used when the issue was encountered yesterday, I'll try to repro it again once rease testing is finished

[cbr7]

@benoitf I've just encountered this issue again, exactly as previously reported I've taken a video of it and have ran the cli commands you mentioned:

vladimirlazar@Vladimirs-MacBook-Pro-2 ~ % podman system connection list
Name URI Identity Default ReadWrite
podman-machine-default ssh://core@127.0.0.1:61641/run/user/501/podman/podman.sock /Users/vladimirlazar/.local/share/containers/podman/machine/machine true true
podman-machine-default-root ssh://root@127.0.0.1:61641/run/podman/podman.sock /Users/vladimirlazar/.local/share/containers/podman/machine/machine false true
podman-machine-rootless ssh://core@127.0.0.1:61578/run/user/501/podman/podman.sock /Users/vladimirlazar/.local/share/containers/podman/machine/machine false true
podman-machine-rootless-root ssh://root@127.0.0.1:61578/run/podman/podman.sock /Users/vladimirlazar/.local/share/containers/podman/machine/machine false true

vladimirlazar@Vladimirs-MacBook-Pro-2 ~ % podman machine inspect
[
{
"ConfigDir": {
"Path": "/Users/vladimirlazar/.config/containers/podman/machine/applehv"
},
"ConnectionInfo": {
"PodmanSocket": {
"Path": "/var/folders/r_/b1q04qyn2mdg9vqdp32ysvqh0000gn/T/podman/podman-machine-default-api.sock"
},
"PodmanPipe": null
},
"Created": "2024-04-26T12:24:56.599655+02:00",
"LastUp": "0001-01-01T00:00:00Z",
"Name": "podman-machine-default",
"Resources": {
"CPUs": 6,
"DiskSize": 93,
"Memory": 11920,
"USBs": []
},
"SSHConfig": {
"IdentityPath": "/Users/vladimirlazar/.local/share/containers/podman/machine/machine",
"Port": 61641,
"RemoteUsername": "core"
},
"State": "running",
"UserModeNetworking": true,
"Rootful": true
}
]

Screen.Recording.2024-04-26.at.14.14.05.mov

[benoitf]

On Podman Desktop side we're not handling HyperV until Podman 5.1 (bugs in podman)

[cbr7]

@benoitf but this is on mac, on applehv, only the initial report was from hyperV

[benoitf]

the title says HyperV so it's hard to follow :-)

[benoitf]

label is OS/windows as well

[cbr7]

Removed the label and changed the name to remove hyperV

[odockal]

I am seeing this again. But this time we are on WSL, Windows and problem truly lies with Podman Desktop not able to pull image while cli works well. This time this all is happening when I have switched between two machine (setting new default connection via podman desktop).

[lstocchi]

Open an issue on podman that could be related to this #6455

[lstocchi]

@odockal i've been trying to replicate it for many times now without any success. Have you seen it while testing 1.11? If not maybe we can close it and reopen when we have a reproducer. If it's a problem with the default connection it would be good to test with #7627

[cbr7]

@lstocchi Yeah it happened to me on Thursday while testing 1.11.

[lstocchi]

@cbr7 do you have any step to reproduce it? Or is it random? I just tried again, no luck. If it's related to the default system connection could you try by using #7627 ?

[cbr7]

@lstocchi there are no special steps to reproduce it, sometimes it just happens that PD enters a state where all the resources related to kind/minikube clusters that were just created from PD resources menu are invisible inside PD, that includes the images, the containers and the clusters that should be visible under resources.

Additionally turning off and on the podman machine does not fix it, the resources will still be invisible in PD, but to know that checking for them on podman CLI shows them to be there and running, so somehow they are only invisble in PD.

I've also identified a heal for this situation, if the user turns off the podman machine and deletes it and after that creates a new podman machine, if at that point another kind/minikube cluster is created from PD this one will be visible inside PD.