Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot connect to debug port on Windows (WSL2) #12292

Closed
voquangphu opened this issue Nov 14, 2021 · 26 comments · Fixed by #14026
Closed

Cannot connect to debug port on Windows (WSL2) #12292

voquangphu opened this issue Nov 14, 2021 · 26 comments · Fixed by #14026
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@voquangphu
Copy link

voquangphu commented Nov 14, 2021

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I have been trying to migrate all Docker recipes to Podman for both Mac and Windows.
On Windows running Debian (WSL 2) I am able to install, build and run Podman container successfully, running Tomcat 7 service and access the service from Windows host (via localhost:8080).
However, I am not able to connect to debug port 8000 inside container for some reasons. (I'm using IntelliJ to debug)
Steps to reproduce the issue:

  1. Run a CentOS 6 base instance with port mapping 8080 and 8000
podman run -td --name centos -p 8080:8080 -p 8000:8000 centos:6.10 bash -l
  1. Install Java 1.8 and Tomcat 7

  2. Setup Catalina option in /usr/share/tomcat/conf/tomcat.conf with the below option

CATALINA_OPTS="-Dlog.host=`hostname -s` -Xms256m -Xmx256m -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+UseG1GC -Djava.security.egd=file:/dev/./urandom -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n"
  1. Start tomcat service using service tomcat start
  2. On Windows host, run IntelliJ, setup remote JVM debug to Attach to remote JVM to localhost:8000
  3. Run the above debug configuration

Describe the results you received:
Error running 'remote-debug': Unable to open debugger port (localhost:8000): java.net.ConnectException "Connection refused: connect"

Describe the results you expected:
Debugger connects to Tomcat instance inside Podman container running on WSL 2.

Additional information you deem important (e.g. issue happens only occasionally):
Some additional things I want to add:

  • I use the same recipe for Podman on Mac running Podman machine VM, and IntelliJ is able to connect to debug port localhost:8000
  • I also tested with Windows Docker Desktop running WSL 2 backend and IntelliJ is also able to connect to debug port localhost:8000
  • I checked if port 8000 is accessible from Windows host using netstat, and both port 8080 (obviously) and port 8000 were listening.
PS C:\Users\christopher.vo> netstat -a | findstr :8000                                                                    TCP    [::1]:8000             P50442:0               LISTENING
PS C:\Users\christopher.vo> netstat -a | findstr :8080                                                                    TCP    [::1]:8080             P50442:0               LISTENING

Output of podman version:

Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.15.9
Built:        Thu Jan  1 09:00:00 1970
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.19.6
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/bin/conmon'
    path: /usr/bin/conmon
    version: 'conmon version 2.0.25, commit: unknown'
  cpus: 4
  distribution:
    distribution: debian
    version: "11"
  eventLogger: journald
  hostname: P50442
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.10.16.3-microsoft-standard-WSL2
  linkmode: dynamic
  memFree: 4746706944
  memTotal: 13265854464
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.17
      commit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /tmp/podman-run-1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    selinuxEnabled: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.0.1
      commit: 6a7b16babc95b6a3056b33fb45b74a6f62262dd4
      libslirp: 4.4.0
  swapFree: 4294967296
  swapTotal: 4294967296
  uptime: 5h 11m 13.78s (Approximately 0.21 days)
registries: {}
store:
  configFile: /home/christopher/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: 'fuse-overlayfs: /usr/bin/fuse-overlayfs'
      Version: |-
        fusermount3 version: 3.10.3
        fuse-overlayfs: version 1.4
        FUSE library version 3.10.3
        using FUSE kernel interface version 7.31
  graphRoot: /home/christopher/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 27
  runRoot: /tmp/podman-run-1000/containers
  volumePath: /home/christopher/.local/share/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 0
  BuiltTime: Thu Jan  1 09:00:00 1970
  GitCommit: ""
  GoVersion: go1.15.9
  OsArch: linux/amd64
  Version: 3.0.1

Package info (e.g. output of rpm -q podman or apt list podman):

Listing... Done
podman/stable,now 3.0.1+dfsg1-3+b2 amd64 [installed]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
Debian on WSL 2 (Windows)

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Nov 14, 2021
@flouthoc
Copy link
Collaborator

Are you able to connect the ones on 8080 ? also could you paste output of podman inspect <your-container>.
If possible could you try switching your debug port from 8000 to something else and try.

@voquangphu
Copy link
Author

yes I am able to access HTTP port 8080, it can render the application page correctly.
I also tried to switch debug port to 8001, 8081 but none of them worked.
Here is the output of podman inspect

[
    {
        "Id": "1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9",
        "Created": "2021-11-15T22:11:37.1728002+09:00",
        "Path": "bash",
        "Args": [
            "-l"
        ],
        "State": {
            "OciVersion": "1.0.2-dev",
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 1532,
            "ConmonPid": 1529,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2021-11-15T22:11:37.6934336+09:00",
            "FinishedAt": "0001-01-01T00:00:00Z",
            "Healthcheck": {
                "Status": "",
                "FailingStreak": 0,
                "Log": null
            }
        },
        "Image": "2c50ff37e58006b3c39cf8c0bdc5035e4ae5b4c09846be7b26b7bcbbba80d714",
        "ImageName": "localhost/ds:latest",
        "Rootfs": "",
        "Pod": "",
        "ResolvConfPath": "/tmp/podman-run-1000/containers/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata/resolv.conf",
        "HostnamePath": "/tmp/podman-run-1000/containers/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata/hostname",
        "HostsPath": "/tmp/podman-run-1000/containers/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata/hosts",
        "StaticDir": "/home/christopher/.local/share/containers/storage/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata",
        "OCIConfigPath": "/home/christopher/.local/share/containers/storage/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata/config.json",
        "OCIRuntime": "crun",
        "ConmonPidFile": "/tmp/podman-run-1000/containers/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata/conmon.pid",
        "PidFile": "/tmp/podman-run-1000/containers/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata/pidfile",
        "Name": "ds-final",
        "RestartCount": 0,
        "Driver": "overlay",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "EffectiveCaps": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FOWNER",
            "CAP_FSETID",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE",
            "CAP_SETFCAP",
            "CAP_SETGID",
            "CAP_SETPCAP",
            "CAP_SETUID",
            "CAP_SYS_CHROOT"
        ],
        "BoundingCaps": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FOWNER",
            "CAP_FSETID",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE",
            "CAP_SETFCAP",
            "CAP_SETGID",
            "CAP_SETPCAP",
            "CAP_SETUID",
            "CAP_SYS_CHROOT"
        ],
        "ExecIDs": [],
        "GraphDriver": {
            "Name": "overlay",
            "Data": {
                "LowerDir": "/home/christopher/.local/share/containers/storage/overlay/e002edc1093e6fe5c6e03123b1fa5ba7792835a049be3a83ebf846d9d7205e15/diff:/home/christopher/.local/share/containers/storage/overlay/ae6fde8d3953a9962f83911861f0d912a645f8f130832a8a92897ad55c7cf425/diff:/home/christopher/.local/share/containers/storage/overlay/4d554cee1be0c7ec497480f43e21023f6bc9ec59d1d46ad665ca66fa1f494384/diff:/home/christopher/.local/share/containers/storage/overlay/823c7f7ba05eb85f4fa9e679ea01ce3dcd2389d6a42916177f068a58b16b59a5/diff:/home/christopher/.local/share/containers/storage/overlay/ebee29b5e5d4bca2baf7669c350c0d10ed7f68790a02dd46e7472d8ab6645346/diff:/home/christopher/.local/share/containers/storage/overlay/d5e85507dd96d5a6c3903d8492be74da27bc2530291ca0971e2f670af132c84c/diff:/home/christopher/.local/share/containers/storage/overlay/a92f1183301b3a90b6bbde6869c237ccfa655f679bd414439f99ed366fb8b0ad/diff:/home/christopher/.local/share/containers/storage/overlay/1aa3461d04343d62d2fced0e12ace5d08ca3501215b1290464306800746dde84/diff:/home/christopher/.local/share/containers/storage/overlay/ec001920c5f59cfce7beea3e5f5c06f9a157d640bcd08779ab2ac6e81ed12837/diff:/home/christopher/.local/share/containers/storage/overlay/e5a0e676b9c71b9ed4230785c3f9df868661aa0dd74ad261f3afb7ac30701e51/diff:/home/christopher/.local/share/containers/storage/overlay/ae9a47162f296c83e2d8f5c291c13379ddf248b7c8f073c8c82bb01cd927b9bb/diff:/home/christopher/.local/share/containers/storage/overlay/ed9bb5dd4a8b62fd005c3520e833dfb7f44ac0be443a1572ebc6ed1100d58f13/diff:/home/christopher/.local/share/containers/storage/overlay/3ddf72510516bd335ee6e8e5ee94d0cda593eae4853665b67a6aeaf8c9619361/diff:/home/christopher/.local/share/containers/storage/overlay/7c8a17f6d7c8983143c4ac664822208392309959130336325bcf55f8a1b70393/diff:/home/christopher/.local/share/containers/storage/overlay/18b4734c179b93596067400f7d62b4e6ae30492c16759e89dea6bb7babed7a8e/diff:/home/christopher/.local/share/containers/storage/overlay/ab09eb42686083e592f56af645c9d86a81bb5403205fa7c475846e337eff4155/diff:/home/christopher/.local/share/containers/storage/overlay/c1601f63912f7c10a6e44d0a7bb3c7a4290eee151c385341715cb363b28cf492/diff:/home/christopher/.local/share/containers/storage/overlay/4ba6c3a91ff0cb1582d9020bb0e86cb2915b1384b58999380aeb7fd3b8eb7b88/diff:/home/christopher/.local/share/containers/storage/overlay/10576818fe08a7ca8818746db42567f5e248b5a0f53bad6c57c6ef03fa8e9879/diff:/home/christopher/.local/share/containers/storage/overlay/a17075254231e48def0e5760529538a975e933b5740720c1988bda45959a1ada/diff:/home/christopher/.local/share/containers/storage/overlay/09eb4b699894f9b2ea54e11b2c787e34f11d0eb8ec24267c2ef1b614a90f3b56/diff:/home/christopher/.local/share/containers/storage/overlay/66e42a50d2582c5dcfd3fc026f022d8851bc6d47a7c40324bca8661f36eae3bf/diff:/home/christopher/.local/share/containers/storage/overlay/7953bbf25c8e202df94432d20eeaad657f801b9a1bc146f7739689752728b257/diff:/home/christopher/.local/share/containers/storage/overlay/612beaca64e7bf2f46ee9070519304c88dec386891c429e7a8a58837843553a9/diff:/home/christopher/.local/share/containers/storage/overlay/bbc9fc6fa895ddccbb9408c455385d84e42540218e53470d397f2b752b8c7a6b/diff:/home/christopher/.local/share/containers/storage/overlay/9f4a72ddfdea24c59968ed55ca925f9535865647e576ca7b7e5be023cb768491/diff:/home/christopher/.local/share/containers/storage/overlay/8088cb617267a10ef6b4d0db6b0ebcedcd9b3af5183d64351b92bea6376ad62c/diff",
                "MergedDir": "/home/christopher/.local/share/containers/storage/overlay/45b09ad9a8f3b2cc1dee9a2a20082a8898c77a7ee160b66d8991f60a95995b9a/merged",
                "UpperDir": "/home/christopher/.local/share/containers/storage/overlay/45b09ad9a8f3b2cc1dee9a2a20082a8898c77a7ee160b66d8991f60a95995b9a/diff",
                "WorkDir": "/home/christopher/.local/share/containers/storage/overlay/45b09ad9a8f3b2cc1dee9a2a20082a8898c77a7ee160b66d8991f60a95995b9a/work"
            }
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/mnt/c/Users/christopher.vo/rakuten/domestic-search/target/ds.war",
                "Destination": "/usr/share/tomcat/webapps/ds.war",
                "Driver": "",
                "Mode": "",
                "Options": [
                    "rbind"
                ],
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Dependencies": [],
        "NetworkSettings": {
            "EndpointID": "",
            "Gateway": "",
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "MacAddress": "",
            "Bridge": "",
            "SandboxID": "",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "8000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8000"
                    }
                ],
                "8080/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8080"
                    }
                ]
            },
            "SandboxKey": "/tmp/podman-run-1000/netns/cni-67e5f331-9230-782c-c129-c3117226c7df"
        },
        "ExitCommand": [
            "/usr/bin/podman",
            "--root",
            "/home/christopher/.local/share/containers/storage",
            "--runroot",
            "/tmp/podman-run-1000/containers",
            "--log-level",
            "warning",
            "--cgroup-manager",
            "cgroupfs",
            "--tmpdir",
            "/tmp/run-1000/libpod/tmp",
            "--runtime",
            "crun",
            "--storage-driver",
            "overlay",
            "--storage-opt",
            "overlay.mount_program=/usr/bin/fuse-overlayfs",
            "--events-backend",
            "file",
            "container",
            "cleanup",
            "1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9"
        ],
        "Namespace": "",
        "IsInfra": false,
        "Config": {
            "Hostname": "1a6187a91d36",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": true,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "TERM=xterm",
                "container=podman",
                "HOME=/root",
                "HOSTNAME=1a6187a91d36"
            ],
            "Cmd": [
                "bash",
                "-l"
            ],
            "Image": "localhost/ds:latest",
            "Volumes": null,
            "WorkingDir": "/",
            "Entrypoint": "",
            "OnBuild": null,
            "Labels": {
                "io.buildah.version": "1.22.3",
                "org.label-schema.build-date": "20180804",
                "org.label-schema.license": "GPLv2",
                "org.label-schema.name": "CentOS Base Image",
                "org.label-schema.schema-version": "1.0",
                "org.label-schema.vendor": "CentOS"
            },
            "Annotations": {
                "io.container.manager": "libpod",
                "io.kubernetes.cri-o.Created": "2021-11-15T22:11:37.1728002+09:00",
                "io.kubernetes.cri-o.TTY": "true",
                "io.podman.annotations.autoremove": "FALSE",
                "io.podman.annotations.init": "FALSE",
                "io.podman.annotations.privileged": "FALSE",
                "io.podman.annotations.publish-all": "FALSE",
                "org.opencontainers.image.stopSignal": "15"
            },
            "StopSignal": 15,
            "CreateCommand": [
                "podman",
                "run",
                "-td",
                "--name",
                "ds-final",
                "-p",
                "8080:8080",
                "-p",
                "8000:8000",
                "-v",
                "/mnt/c/Users/christopher.vo/rakuten/domestic-search/target/ds.war:/usr/share/tomcat/webapps/ds.war",
                "ds",
                "bash",
                "-l"
            ],
            "Umask": "0022",
            "Timeout": 0,
            "StopTimeout": 10
        },
        "HostConfig": {
            "Binds": [
                "/mnt/c/Users/christopher.vo/rakuten/domestic-search/target/ds.war:/usr/share/tomcat/webapps/ds.war:rw,rprivate,rbind"
            ],
            "CgroupManager": "cgroupfs",
            "CgroupMode": "host",
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "k8s-file",
                "Config": null,
                "Path": "/home/christopher/.local/share/containers/storage/overlay-containers/1a6187a91d36cb329558c26f0f679b55aa72f269cb0257ad030c2e71c236afb9/userdata/ctr.log",
                "Tag": "",
                "Size": "0B"
            },
            "NetworkMode": "slirp4netns",
            "PortBindings": {
                "8000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8000"
                    }
                ],
                "8080/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8080"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": [],
            "CapDrop": [
                "CAP_AUDIT_WRITE",
                "CAP_MKNOD",
                "CAP_NET_RAW"
            ],
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": [],
            "GroupAdd": [],
            "IpcMode": "private",
            "Cgroup": "",
            "Cgroups": "default",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "private",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [],
            "Tmpfs": {},
            "UTSMode": "private",
            "UsernsMode": "",
            "ShmSize": 65536000,
            "Runtime": "oci",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": 0,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": [
                {
                    "Name": "RLIMIT_NOFILE",
                    "Soft": 4096,
                    "Hard": 4096
                },
                {
                    "Name": "RLIMIT_NPROC",
                    "Soft": 32768,
                    "Hard": 32768
                }
            ],
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "CgroupConf": null
        }
    }
]

@mheon
Copy link
Member

mheon commented Nov 15, 2021

Can you verify that port 8000 is working inside the container? My suspicion is more with Tomcat than with Podman given what you've described.

@voquangphu
Copy link
Author

I could verify from Tomcat log that it started service for port 8000 as well as port 8080 from catalina log.
Also as I mentioned, I used exactly the same recipe for Mac and Windows Docker Desktop, both of them can connect to 8000.

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Dec 17, 2021

@voquangphu By any chance have you updated to the latest Podman and latest podman machine (OS)

@davdr
Copy link

davdr commented Dec 17, 2021

From the Windows host, can you check if http://[::1]:8000 works? I suspect it will work and only http://127.0.0.1:8000 fails, which is by default what localhost:8000 is.

@voquangphu
Copy link
Author

I asked the same question in WSL 2 and this solution works, I'll try to see if http://[::1]:8000 also works when I have sometime.
microsoft/WSL#7732

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Jan 18, 2022

Is this still an open issue, or have you found a solution @voquangphu

@dyskette
Copy link

From the Windows host, can you check if http://[::1]:8000 works? I suspect it will work and only http://127.0.0.1:8000 fails, which is by default what localhost:8000 is.

Yeah, that works. I used it for a sql server podman like this: [::1],1433

@davdr
Copy link

davdr commented Feb 9, 2022

This is not a Podman problem but a WSL2 problem: WSL2 treats a socket listening as "*:8080" (= accepting any IPv4 or IPv6 connection) on Linux side in an unexpected way by only binding to an IPv6 socket on the Windows side (see also microsoft/WSL#4851).

  • You will probably only encounter this when you have an application in Windows side that can only resolve "localhost" to the IPv4 127.0.0.1 address (e.g. older Java versions), or when you explicitly try to connect to 127.0.0.1 from the Windows side.

  • Due to a difference in the way Podman and Docker bind their listening sockets, only Podman is affected by this problem, Docker is not:

    • Rootless Podman's containers-rootlessport is listening on a dual-stack IPv4+IPv6 socket by default if no explicit address is specified in the port mapping, and WSL2's wslhost process in that case doesn't bind to the IPv4 localhost address 127.0.0.1 on Windows side, but only to the IPv6 localhost ::1 address.

    • Docker CE behaves differently and by default opens a separate IPv4 and IPv6 socket.

The workaround is to let Podman explicitly open a listening IPv4 socket by using a port mapping like -p 127.0.0.1:exposedport:containerport instead of just -p exposedport:containerport.

Example (executed in Powershell on Windows, using a Debian sid based WSL2 distro and Podman from its packages):

PS C:\dev> wsl -d debian-sid podman version
Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.17.5
OS/Arch:      linux/amd64
PS C:\dev> wsl -d debian-sid uname -a
Linux 2001L4DE2C4 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 GNU/Linux
PS C:\dev> wsl -d debian-sid podman run -it --rm -d -p 8080:80 --name web8080 nginx:1.21
PS C:\dev> wsl -d debian-sid podman run -it --rm -d -p 127.0.0.1:8081:80 --name web8081 nginx:1.21
PS C:\dev> wsl -d debian-sid ss -nltp
State      Recv-Q    Send-Q    Local Address:Port    Peer Address:Port   Process
LISTEN     0         4096                  *:8080               *:*      users:(("exe",pid=947,fd=12))
LISTEN     0         4096          127.0.0.1:8081         0.0.0.0:*      users:(("exe",pid=1076,fd=12))
PS C:\dev> netstat -aon | findstr :808
  TCP    [::1]:8080             [::]:0                 LISTENING       21948
  TCP    127.0.0.1:8081         0.0.0.0:0              LISTENING       21948
PS C:\dev> Invoke-WebRequest -UseBasicParsing http://127.0.0.1:8080
# fails
PS C:\dev> Invoke-WebRequest -UseBasicParsing http://127.0.0.1:8081
# works

As you can see, Podman's *:8080 incorrectly has become [::1]:8080 on Windows side.

If you'd do the same with Docker, you'd see that with -p 8080:80 they open two separate sockets 0.0.0.0:8080 and [::]:8080, which become 127.0.0.1:8080 and [::1]:8080 respectively on Windows side, so no problem there.

@rhatdan is this difference with Docker by design, or unintentional?

@mheon
Copy link
Member

mheon commented Feb 14, 2022

This definitely sounds like a bug, though I'm honestly quite tempted to call it a bug in WSL because this breaks decades of Unix conventions.

Still, if there's a workaround to convince WSL to bind only to v4 addresses in these cases, I'm not opposed to adding it, it won't hurt and it seems like it could be a while before the WSL problem is resolved.

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@iwt-kschoenrock
Copy link

I'm encountering this problem as well (Win 11 21h2)

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Apr 23, 2022

Since this is described as a WSL2 bug, I don't see Podman doing much to fix it. So I am closing.

@n1hility WDYT?

@rhatdan rhatdan closed this as completed Apr 23, 2022
@n1hility
Copy link
Member

n1hility commented Apr 24, 2022

@rhatdan hmm since this works for rootfull and not rootless I wonder if we should change rootlessport to have a setting we can trigger for WSL configs that would either force wildcard to tcp4 (WSL doesn't support fully ipv6 anyway) or setup a tcp4 and tcp6 listener vs the the dual-stack sock. It's technically something we shouldn't have to do, but might remove a usability barrier. I'll ponder some more on this.

@davdr
Copy link

davdr commented Apr 24, 2022

+1 @n1hility

The thing is that Docker works that way, and therefore doesn't trigger the problem. For WSL users trying to migrate to Podman this comes as a surprise when they see something "that works with Docker now doesn't work with Podman", from their point of view.

@n1hility
Copy link
Member

/reopen
/assign

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Apr 27, 2022

@n1hility: Reopened this issue.

In response to this:

/reopen
/assign

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@n1hility
Copy link
Member

Hi everyone, the fix for this will be included in the 4.1 release, and won't require any special properties. You will need to reinit your machine after installing 4.1 [when released] to get the update (podman machine rm; podman machine init.) In the meantime you can use the 127.0.01 workaround above, or you can switch to rootfull.

Thank you @davdr for identifying and calling out the root cause.

@davdr
Copy link

davdr commented May 20, 2022

@n1hility does it require podman machine or any other conditions to work? I have Debian Sid with Podman upgraded to 4.1 (4.1.0+ds2-2 via https://packages.debian.org/experimental/podman), and behavour doesn't seem to have changed (did a podman system reset to get the new network stack): rootlessport is still opening a dual-stack socket.

@n1hility
Copy link
Member

@n1hility does it require podman machine or any other conditions to work? I have Debian Sid with Podman upgraded to 4.1 (4.1.0+ds2-2 via https://packages.debian.org/experimental/podman), and behavour doesn't seem to have changed (did a podman system reset to get the new network stack): rootlessport is still opening a dual-stack socket.

@davdr Yes it does rely on machine integration, but you can trigger it manually by creating “ /etc/containers/podman-machine” with the content of “wsl” (quotes excluded)

@n1hility
Copy link
Member

(Rootlessport looks for that file to decide how to bind)

@davdr
Copy link

davdr commented May 20, 2022

@davdr Yes it does rely on machine integration, but you can trigger it manually by creating “/etc/containers/podman-machine” with the content of “wsl” (quotes excluded)

@n1hility many thanks, works as a charm!

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 20, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants