Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No machine id when using Docker Desktop #4

Closed
FelixMertin opened this issue Mar 31, 2023 · 29 comments
Closed

No machine id when using Docker Desktop #4

FelixMertin opened this issue Mar 31, 2023 · 29 comments

Comments

@FelixMertin
Copy link

FelixMertin commented Mar 31, 2023

Hi,
unfortunately, py-machineid fails to retrieve an id when you are using Docker Desktop. The system where I found this issue was a Apple Macbook 2022 with Docker Desktop.
It seems as cat /proc/self/cgroup only returns 0::/. A fix, more like a workaround, is to start the docker container by docker run --cgroupdns=host but this is not sustainable for my project. As the idea is that the users start the container only with docker run . Issue is discussed here as well.
What is still usable in containers running via Docker Desktop is the /proc/self/mountinfo.

@ezekg
Copy link
Member

ezekg commented Mar 31, 2023

This should already be handled here in #3, but maybe it's broken?

if not id:
mountinfo = __read__('/proc/self/mountinfo')
if mountinfo:
if 'docker' in mountinfo:
id = __exec__("grep 'systemd' /proc/self/mountinfo | cut -d/ -f3")

Let me know if you're open to a PR fixing this bug.

@FelixMertin
Copy link
Author

Yes it is broken. Unfortunately, the id variable is set by line 81 but to the value of 0::/. There is a way of validating for an actual docker id missing.

@ezekg
Copy link
Member

ezekg commented Mar 31, 2023

Are you open to a PR fixing this? 0::/ should not be a valid ID, so that should be set to null if there is no matching ID.

cc @juanbretti

@FelixMertin
Copy link
Author

The current prioritization prevents me from getting involved via PR. The option /proc/self/mountinfo , when testing native inside of container, was also not delivering usable results.

@ezekg
Copy link
Member

ezekg commented Mar 31, 2023

What version of Docker Engine are you running? I can't reproduce your issue with Docker Desktop.

@ezekg
Copy link
Member

ezekg commented Mar 31, 2023

Can you also please provide a Docker container’s /proc/self/cgroup and /proc/self/mountinfo contents?

@ezekg
Copy link
Member

ezekg commented Apr 12, 2023

Let me know if this is still an issue.

@ezekg
Copy link
Member

ezekg commented Jun 23, 2023

Closing due to no response. Please let me know if the issue is still happening and I'll reopen.

@ezekg ezekg closed this as completed Jun 23, 2023
@YiuRULE
Copy link

YiuRULE commented Sep 8, 2023

I have a similar problem here, the one-liner uses to reproduce the problem:

$ docker run python:3.10 /bin/bash -c "pip install py-machineid && python -c 'import machineid; machineid.id()'"

The content of /proc/self/cgroup from the container:

$ cat /proc/self/cgroup
0::/

The content of /proc/self/mountinfo from the container:

$ cat /proc/self/mountinfo
1669 1583 0:165 / / rw,relatime master:583 - overlay overlay rw,lowerdir=/var/lib/docker/overlay2/l/RMQ77OFADZSOSA4FHRVZVM4VBJ:/var/lib/docker/overlay2/l/6VVIVCEMQHI3SRLLTEPWKQNFWF:/var/lib/docker/overlay2/l/5GGA2L6J44HPF5XNGRIJUQACTX:/var/lib/docker/overlay2/l/LTJ3PWBSKOYC4KS4KI2AHJO56F:/var/lib/docker/overlay2/l/P7NSIG2DSNJD7JVW2S7JZ64NMK:/var/lib/docker/overlay2/l/HQ6OCC7EBB3ZI3Z7FO6MOGNSKV:/var/lib/docker/overlay2/l/SFB4GJKHZD4C765SRBXGKPRACT:/var/lib/docker/overlay2/l/KRNPBAIB2VDOGNNDDXM57NVOI6:/var/lib/docker/overlay2/l/KWFOU5AUOFOATLZ7KGLH6VDKHW,upperdir=/var/lib/docker/overlay2/a6de03ea164fee802e155aa3ee569e72af27dbc7c78b49cb9913e0f186c8cb87/diff,workdir=/var/lib/docker/overlay2/a6de03ea164fee802e155aa3ee569e72af27dbc7c78b49cb9913e0f186c8cb87/work
1670 1669 0:181 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
1671 1669 0:182 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1672 1671 0:183 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
1673 1669 0:184 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs ro
1674 1673 0:28 / /sys/fs/cgroup ro,nosuid,nodev,noexec,relatime - cgroup2 cgroup rw,nsdelegate,memory_recursiveprot
1675 1671 0:169 / /dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
1676 1671 0:185 / /dev/shm rw,nosuid,nodev,noexec,relatime - tmpfs shm rw,size=65536k
1677 1669 259:2 /var/lib/docker/containers/a20ed72609228544b1aa217b9fa84b3ff06b09c209dc201c077075563cab06bb/resolv.conf /etc/resolv.conf rw,relatime - ext4 /dev/nvme0n1p2 rw,errors=remount-ro
1678 1669 259:2 /var/lib/docker/containers/a20ed72609228544b1aa217b9fa84b3ff06b09c209dc201c077075563cab06bb/hostname /etc/hostname rw,relatime - ext4 /dev/nvme0n1p2 rw,errors=remount-ro
1679 1669 259:2 /var/lib/docker/containers/a20ed72609228544b1aa217b9fa84b3ff06b09c209dc201c077075563cab06bb/hosts /etc/hosts rw,relatime - ext4 /dev/nvme0n1p2 rw,errors=remount-ro
1597 1671 0:183 /0 /dev/console rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
1620 1670 0:181 /bus /proc/bus ro,nosuid,nodev,noexec,relatime - proc proc rw
1621 1670 0:181 /fs /proc/fs ro,nosuid,nodev,noexec,relatime - proc proc rw
1622 1670 0:181 /irq /proc/irq ro,nosuid,nodev,noexec,relatime - proc proc rw
1623 1670 0:181 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw
1624 1670 0:181 /sysrq-trigger /proc/sysrq-trigger ro,nosuid,nodev,noexec,relatime - proc proc rw
1625 1670 0:187 / /proc/asound ro,relatime - tmpfs tmpfs ro
1626 1670 0:188 / /proc/acpi ro,relatime - tmpfs tmpfs ro
1627 1670 0:182 /null /proc/kcore rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1628 1670 0:182 /null /proc/keys rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1629 1670 0:182 /null /proc/timer_list rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1630 1670 0:182 /null /proc/sched_debug rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1631 1673 0:189 / /sys/firmware ro,relatime - tmpfs tmpfs ro

docker version:

$ docker version
Client: Docker Engine - Community
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:28:22 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:52:17 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

@ezekg
Copy link
Member

ezekg commented Sep 8, 2023

@YiuRULE works fine for me:

$ docker run python:3.10 /bin/bash -c "pip install py-machineid && python -c 'import machineid; print(machineid.id())'"
# Collecting py-machineid
#   Downloading py_machineid-0.4.3-py3-none-any.whl (4.4 kB)
# Installing collected packages: py-machineid
# Successfully installed py-machineid-0.4.3
# WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package # manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
# 
# [notice] A new release of pip is available: 23.0.1 -> 23.2.1
# [notice] To update, run: pip install --upgrade pip
#
# 6d68e6a3b0b3ccde00cf7138a03115936b7bf6cbb707f6fbc0babad1fd403395

See the hash at the end. Your original reproduction command did not print anything, by the way.

Client: Docker Engine - Community
 Cloud integration: v1.0.31
 Version:           23.0.5
 API version:       1.42
 Go version:        go1.19.8
 Git commit:        bc4487a
 Built:             Wed Apr 26 16:17:45 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Desktop
 Engine:
  Version:          23.0.5
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       94d3ad6
  Built:            Wed Apr 26 16:17:45 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Open to a PR if you can determine a fix.

@YiuRULE
Copy link

YiuRULE commented Sep 8, 2023

Here the detailed output if needed:

$ docker run python:3.10 /bin/bash -c "pip install py-machineid && python -c 'import machineid; machineid.id()'"
Collecting py-machineid
  Downloading py_machineid-0.4.3-py3-none-any.whl (4.4 kB)
Installing collected packages: py-machineid
Successfully installed py-machineid-0.4.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/machineid/__init__.py", line 99, in id
    raise Exception('failed to obtain id on platform {}'.format(platform))
Exception: failed to obtain id on platform linux

@ezekg
Copy link
Member

ezekg commented Sep 8, 2023

Would you be open to a PR? Starting point:

if not id:
cgroup = __read__('/proc/self/cgroup')
if cgroup:
if 'docker' in cgroup:
id = __exec__('head -1 /proc/self/cgroup | cut -d/ -f3')
if not id:
mountinfo = __read__('/proc/self/mountinfo')
if mountinfo:
if 'docker' in mountinfo:
id = __exec__("grep 'systemd' /proc/self/mountinfo | cut -d/ -f3")

Looks like your mountinfo doesn't have systemd, so maybe we should try a different keyword too.

@ezekg ezekg reopened this Sep 8, 2023
@ezekg
Copy link
Member

ezekg commented Sep 8, 2023

For reference, here's my mountinfo:

1721 1538 0:328 / / rw,relatime master:332 - overlay overlay rw,lowerdir=/var/lib/docker/overlay2/l/ULPQKKHZTOVX4T6VTB7YNLG6GX:/var/lib/docker/overlay2/l/LKY5HLEDPMRZ4YZOMW35P4RO3K:/var/lib/docker/overlay2/l/JV4JIU2SK7X5E2FF6DPCWF3ATI:/var/lib/docker/overlay2/l/MASRPIMP3PFJFUW5IH7YGFQCVC:/var/lib/docker/overlay2/l/HMERM4ZTMYX33B4KPBBVIRHMSK:/var/lib/docker/overlay2/l/KJ3MIV6OOWSAZP7XLYYQQL4B3N:/var/lib/docker/overlay2/l/4KNC4IPMXFJSOICSDMC55YZOXM:/var/lib/docker/overlay2/l/B3D7IKWVFYRPIXU6LLEEQKK6CC:/var/lib/docker/overlay2/l/CBSGFHEACFHHDPORYRS2K2FEQE,upperdir=/var/lib/docker/overlay2/9ce3d5f3b6dbc42ac75e823cd7a21b2d76438c26fc52d9d2b26b101c5b3dbe69/diff,workdir=/var/lib/docker/overlay2/9ce3d5f3b6dbc42ac75e823cd7a21b2d76438c26fc52d9d2b26b101c5b3dbe69/work
1722 1721 0:331 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
1723 1721 0:332 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1724 1723 0:333 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
1725 1721 0:334 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs ro
1726 1725 0:335 / /sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,mode=755
1727 1726 0:56 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/cpuset ro,nosuid,nodev,noexec,relatime master:79 - cgroup cpuset rw,cpuset
1728 1726 0:57 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/cpu ro,nosuid,nodev,noexec,relatime master:80 - cgroup cpu rw,cpu
1729 1726 0:58 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/cpuacct ro,nosuid,nodev,noexec,relatime master:81 - cgroup cpuacct rw,cpuacct
1730 1726 0:59 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/blkio ro,nosuid,nodev,noexec,relatime master:82 - cgroup blkio rw,blkio
1731 1726 0:60 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/memory ro,nosuid,nodev,noexec,relatime master:83 - cgroup memory rw,memory
1732 1726 0:61 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/devices ro,nosuid,nodev,noexec,relatime master:84 - cgroup devices rw,devices
1733 1726 0:62 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/freezer ro,nosuid,nodev,noexec,relatime master:85 - cgroup freezer rw,freezer
1734 1726 0:63 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/net_cls ro,nosuid,nodev,noexec,relatime master:86 - cgroup net_cls rw,net_cls
1735 1726 0:64 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/perf_event ro,nosuid,nodev,noexec,relatime master:87 - cgroup perf_event rw,perf_event
1736 1726 0:65 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/net_prio ro,nosuid,nodev,noexec,relatime master:88 - cgroup net_prio rw,net_prio
1737 1726 0:66 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/hugetlb ro,nosuid,nodev,noexec,relatime master:89 - cgroup hugetlb rw,hugetlb
1738 1726 0:67 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/pids ro,nosuid,nodev,noexec,relatime master:90 - cgroup pids rw,pids
1739 1726 0:68 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/rdma ro,nosuid,nodev,noexec,relatime master:91 - cgroup rdma rw,rdma
1740 1726 0:69 / /sys/fs/cgroup/misc ro,nosuid,nodev,noexec,relatime master:92 - cgroup misc rw,misc
1741 1726 0:144 /docker/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3 /sys/fs/cgroup/systemd ro,nosuid,nodev,noexec,relatime master:93 - cgroup cgroup rw,name=systemd
1742 1723 0:330 / /dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
1743 1723 0:336 / /dev/shm rw,nosuid,nodev,noexec,relatime - tmpfs shm rw,size=65536k
1744 1721 8:64 /data/docker/containers/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3/resolv.conf /etc/resolv.conf rw,relatime - ext4 /dev/sde rw,discard,errors=remount-ro,data=ordered
1745 1721 8:64 /data/docker/containers/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3/hostname /etc/hostname rw,relatime - ext4 /dev/sde rw,discard,errors=remount-ro,data=ordered      
1746 1721 8:64 /data/docker/containers/b43219447f2d8a9c1ab3bd7781899b686a8a4aa0d51101f8437adf8abf7753a3/hosts /etc/hosts rw,relatime - ext4 /dev/sde rw,discard,errors=remount-ro,data=ordered
1572 1722 0:331 /bus /proc/bus ro,nosuid,nodev,noexec,relatime - proc proc rw
1573 1722 0:331 /fs /proc/fs ro,nosuid,nodev,noexec,relatime - proc proc rw
1574 1722 0:331 /irq /proc/irq ro,nosuid,nodev,noexec,relatime - proc proc rw
1575 1722 0:331 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw
1576 1722 0:337 / /proc/acpi ro,relatime - tmpfs tmpfs ro
1577 1722 0:332 /null /proc/kcore rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1578 1722 0:332 /null /proc/keys rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1579 1722 0:332 /null /proc/timer_list rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
1580 1725 0:338 / /sys/firmware ro,relatime - tmpfs tmpfs ro

@ezekg
Copy link
Member

ezekg commented Sep 8, 2023

Maybe we can try grep 'resolv.conf' /proc/self/mountinfo | cut -d/ -f6 if the systemd grep comes up empty?

That seems to work for your mountinfo.

@ezekg
Copy link
Member

ezekg commented Sep 8, 2023

Better yet, maybe we should grep on /data/docker/containers and then /var/lib/docker/containers instead? That way we can make sure we don't get false positives, e.g. grep 'resolv.conf' /proc/self/mountinfo | cut -d/ -f6 on my mountinfo gives an invalid machine ID "resolve.conf" (because my ID is -f5 not -f6).

@ezekg
Copy link
Member

ezekg commented Oct 5, 2023

I pushed f079039, which should resolve this issue. Can everyone please test the master branch?

docker run python:3.12 /bin/bash -c "pip install git+https://github.com/keygen-sh/py-machineid && python -c 'import machineid; print(machineid.id())'"
# => a3253158467771c17587282c2227440903a49639ccada8bbc536bf6c0ff97126

Aside: if anybody has tips for improving tests (mocking file system?), I'm all ears. 🙂

@juanbretti
Copy link
Contributor

Hello @ezekg .
I tested your proposal and it does not work for me.

Here are my tests on my dockers:

root@docker_machine:/# grep 'systemd' /proc/self/mountinfo | cut -d/ -f3
1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx7
root@docker_machine:/# grep '/data/docker/containers' /proc/self/mountinfo | cut -d/ -f5
root@docker_machine:/# grep '/var/lib/docker/containers' /proc/self/mountinfo | cut -d/ -f6

As you can see, the second and third, are not pulling any result.
Maybe you could leave the first option, as an alternative if the other are not pulling any id.

@ezekg
Copy link
Member

ezekg commented Oct 5, 2023

@juanbretti what version of Docker? Can you please share your /proc/self/mountinfo so that I can test against it?

Regardless, I readded the systemd grep in 902823f.

@juanbretti
Copy link
Contributor

Here is the content of my /proc/self/mountinfo.

239 80 0:30 /@docker/btrfs/subvolumes/1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx5 / rw,nodev,relatime master:1 - btrfs /dev/vg1000/lv rw,ssd,synoacl,nospace_cache,auto_reclaim_space,metadata_ratio=50,syno_allocator
240 239 0:603 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
241 239 0:629 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
242 241 0:630 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
243 239 0:631 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs ro
244 243 0:632 / /sys/fs/cgroup rw,nosuid,nodev,noexec,relatime - tmpfs tmpfs rw,mode=755
245 244 0:17 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/systemd ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
246 244 0:18 /system.slice/pkg-ContainerManager-dockerd.service /sys/fs/cgroup/synomonitor ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,name=synomonitor
247 244 0:20 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/cpuset ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,cpuset
248 244 0:21 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/cpuacct ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,cpuacct
249 244 0:22 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/memory ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,memory
250 244 0:23 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/cpu ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,cpu
251 244 0:24 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/freezer ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,freezer
252 244 0:25 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/blkio ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,blkio
253 244 0:26 /docker/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6 /sys/fs/cgroup/devices ro,nosuid,nodev,noexec,relatime - cgroup cgroup rw,devices
254 241 0:602 / /dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
255 241 0:633 / /dev/shm rw,nosuid,nodev,noexec,relatime - tmpfs shm rw,size=65536k
256 239 0:30 /docker/docker_machine /usr/src rw,nodev,relatime - btrfs /dev/vg1000/lv rw,ssd,synoacl,nospace_cache,auto_reclaim_space,metadata_ratio=50,syno_allocator
257 239 0:663 /docker/docker_machine /usr/storage rw,relatime - ecryptfs /volume1/@storage@ rw,ecryptfs_fnek_sig=1xxxxxxxxxxxxxx2,ecryptfs_sig=1xxxxxxxxxxxxxx2,ecryptfs_cipher=aes,ecryptfs_key_bytes=32,ecryptfs_unlink_sigs
258 239 0:30 /@docker/containers/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6/resolv.conf /etc/resolv.conf rw,nodev,relatime - btrfs /dev/vg1000/lv rw,ssd,synoacl,nospace_cache,auto_reclaim_space,metadata_ratio=50,syno_allocator
259 239 0:30 /@docker/containers/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6/hostname /etc/hostname rw,nodev,relatime - btrfs /dev/vg1000/lv rw,ssd,synoacl,nospace_cache,auto_reclaim_space,metadata_ratio=50,syno_allocator
260 239 0:30 /@docker/containers/2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6/hosts /etc/hosts rw,nodev,relatime - btrfs /dev/vg1000/lv rw,ssd,synoacl,nospace_cache,auto_reclaim_space,metadata_ratio=50,syno_allocator
118 241 0:630 /0 /dev/console rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
119 240 0:603 /bus /proc/bus ro,nosuid,nodev,noexec,relatime - proc proc rw
120 240 0:603 /fs /proc/fs ro,nosuid,nodev,noexec,relatime - proc proc rw
121 240 0:603 /irq /proc/irq ro,nosuid,nodev,noexec,relatime - proc proc rw
122 240 0:603 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw
123 240 0:603 /sysrq-trigger /proc/sysrq-trigger ro,nosuid,nodev,noexec,relatime - proc proc rw
124 240 0:634 / /proc/acpi ro,relatime - tmpfs tmpfs ro
125 240 0:629 /null /proc/kcore rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
126 240 0:629 /null /proc/keys rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
127 240 0:629 /null /proc/timer_list rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
128 240 0:629 /null /proc/sched_debug rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755
157 240 0:635 / /proc/scsi ro,relatime - tmpfs tmpfs ro
158 243 0:636 / /sys/firmware ro,relatime - tmpfs tmpfs ro

And the version of my Docker:

Docker version 20.10.23, build 876964a

@ezekg
Copy link
Member

ezekg commented Oct 5, 2023

@juanbretti I updated master to grep on /@docker/containers instead of systemd so that it's less brittle in the future.

@juanbretti
Copy link
Contributor

Looks good to me!

@ezekg
Copy link
Member

ezekg commented Oct 5, 2023

I've tested against all 3 variants of /proc/self/mountinfo and they return the correct fingerprint. So I'm going to close this in hopes that the problem is resolved. I just published v0.4.4. Please let me know if you find any issues.

@ezekg ezekg closed this as completed Oct 5, 2023
@LucaNicosia
Copy link

Hi team.
I'm interested in using this tool to fingerprint containers, but I'm having the same issue reported here.

docker run python:3.11.6 /bin/bash -c "pip install py-machineid && python -c 'import machineid; machineid.id()'"
Collecting py-machineid
  Obtaining dependency information for py-machineid from https://files.pythonhosted.org/packages/66/b0/613422fb18d26991813c37dcf41f3da7f50d1233cccd1bab5ef280f0d322/py_machineid-0.4.4-py3-none-any.whl.metadata
  Downloading py_machineid-0.4.4-py3-none-any.whl.metadata (1.8 kB)
Downloading py_machineid-0.4.4-py3-none-any.whl (4.5 kB)
Installing collected packages: py-machineid
Successfully installed py-machineid-0.4.4
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: pip install --upgrade pip
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/site-packages/machineid/__init__.py", line 103, in id
    raise Exception('failed to obtain id on platform {}'.format(platform))
Exception: failed to obtain id on platform linux
docker --version                                                                                                                                                                                 
Docker version 24.0.7, build afdd53b

I saw all the previous tests were performed on docker 20 and 23. Maybe the issue is only happening with the newer docker versions?

I'm happy to give you any additional information you need to solve this issue

@ezekg
Copy link
Member

ezekg commented Oct 31, 2023

@LucaNicosia please share the contents of /proc/self/mountinfo from a failing Docker container.

I'm using Docker v24.0.6 and it works fine. Curious what would be causing this.

@LucaNicosia
Copy link

Thanks @ezekg for the quick answer.
Here the output of the command you asked

17512 13668 0:755 / / rw,relatime master:3651 - overlay overlay rw,lowerdir=/var/lib/docker/overlay2/l/TZJS3GP4RQCWZJYZPSRYIS5WBQ:/var/lib/docker/overlay2/l/GPSDLROM4FKAG32N7QVZYXJMWL:/var/lib/docker/overlay2/l/6HOYFNKZ4BHMSZ7LWKM7QFVPLD:/var/lib/docker/overlay2/l/B2VVBMPESRJGBAB5LJRMTPBW55:/var/lib/docker/overlay2/l/ZF4L67RZEBTTZ45UQ7KEN7JFZQ:/var/lib/docker/overlay2/l/MCCDH5IPBDM36SLRFAAJF6CQGV:/var/lib/docker/overlay2/l/P7DLHXASRUNW37K7KTKQQW2NX7:/var/lib/docker/overlay2/l/XRM3F6RUERLKCK3S7NVCLWGE4L:/var/lib/docker/overlay2/l/MCY3UVNXFXJFVWAHCSXNTAB3OY,upperdir=/var/lib/docker/overlay2/fee039a2b1cbb77bfecf3ac5c6ebe276fe44a765ed3b01070d72821b3f015543/diff,workdir=/var/lib/docker/overlay2/fee039a2b1cbb77bfecf3ac5c6ebe276fe44a765ed3b01070d72821b3f015543/work,nouserxattr
17513 17512 0:759 / /proc rw,nosuid,nodev,noexec,relatime - proc proc rw
17514 17512 0:760 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
17515 17514 0:761 / /dev/pts rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
17516 17512 0:762 / /sys ro,nosuid,nodev,noexec,relatime - sysfs sysfs ro
17517 17516 0:30 / /sys/fs/cgroup ro,nosuid,nodev,noexec,relatime - cgroup2 cgroup rw
17518 17514 0:758 / /dev/mqueue rw,nosuid,nodev,noexec,relatime - mqueue mqueue rw
17519 17514 0:763 / /dev/shm rw,nosuid,nodev,noexec,relatime - tmpfs shm rw,size=65536k,inode64
17520 17512 259:2 /lib/docker/containers/ccafdfa582c308c87328801eea67a1c380eb447abfaa7b9c04beea6b60be8dec/resolv.conf /etc/resolv.conf rw,relatime - ext4 /dev/nvme0n1p2 rw
17521 17512 259:2 /lib/docker/containers/ccafdfa582c308c87328801eea67a1c380eb447abfaa7b9c04beea6b60be8dec/hostname /etc/hostname rw,relatime - ext4 /dev/nvme0n1p2 rw
17522 17512 259:2 /lib/docker/containers/ccafdfa582c308c87328801eea67a1c380eb447abfaa7b9c04beea6b60be8dec/hosts /etc/hosts rw,relatime - ext4 /dev/nvme0n1p2 rw
13669 17514 0:761 /0 /dev/console rw,nosuid,noexec,relatime - devpts devpts rw,gid=5,mode=620,ptmxmode=666
13670 17513 0:759 /bus /proc/bus ro,nosuid,nodev,noexec,relatime - proc proc rw
13671 17513 0:759 /fs /proc/fs ro,nosuid,nodev,noexec,relatime - proc proc rw
13672 17513 0:759 /irq /proc/irq ro,nosuid,nodev,noexec,relatime - proc proc rw
13673 17513 0:759 /sys /proc/sys ro,nosuid,nodev,noexec,relatime - proc proc rw
13674 17513 0:759 /sysrq-trigger /proc/sysrq-trigger ro,nosuid,nodev,noexec,relatime - proc proc rw
13675 17513 0:764 / /proc/asound ro,relatime - tmpfs tmpfs ro,inode64
13676 17513 0:765 / /proc/acpi ro,relatime - tmpfs tmpfs ro,inode64
13677 17513 0:760 /null /proc/kcore rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
13678 17513 0:760 /null /proc/keys rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
13679 17513 0:760 /null /proc/timer_list rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64
13680 17513 0:766 / /proc/scsi ro,relatime - tmpfs tmpfs ro,inode64
13689 17516 0:767 / /sys/firmware ro,relatime - tmpfs tmpfs ro,inode64
13880 17516 0:768 / /sys/devices/virtual/powercap ro,relatime - tmpfs tmpfs ro,inode64

@juanbretti
Copy link
Contributor

Mybe, we could use something like this.

    if not id:
      mountinfo = __read__('/proc/self/mountinfo')
      if mountinfo:
        if 'docker' in mountinfo:
          id = __exec__("grep '/var/lib/docker/containers/.*/hostname' /proc/self/mountinfo | cut -d/ -f6")
          if not id:
            id = __exec__("grep '/data/docker/containers/.*/hostname' /proc/self/mountinfo | cut -d/ -f5")
          if not id:
            id = __exec__("grep '/@docker/containers/.*/hostname' /proc/self/mountinfo | cut -d/ -f4")
          if not id:
            id = __exec__("grep -oP '(?<=/)([a-f0-9]+)(?=/hostname)' /proc/self/mountinfo")

@ezekg
Copy link
Member

ezekg commented Nov 2, 2023

@juanbretti open to a PR? What is we rolled all this up into the following?

grep -oP '(?<=docker/containers/)([a-f0-9]+)(?=/hostname)' /proc/self/mountinfo

That'd reduce false matches if something with a /hostname line was ever present that isn't container-specific.

@ezekg ezekg reopened this Nov 2, 2023
@ezekg
Copy link
Member

ezekg commented Nov 2, 2023

Released v0.4.5 that offers a wider support for /proc/self/mountinfo formats. Please test with the following:

docker run python:3.11.6 /bin/bash -c "pip install py-machineid==0.4.5 && python -c 'import machineid; print(machineid.id())'"

Thanks for the starting point, @juanbretti.

@ezekg ezekg closed this as completed Nov 2, 2023
@LucaNicosia
Copy link

Released v0.4.5 that offers a wider support for /proc/self/mountinfo formats. Please test with the following:

docker run python:3.11.6 /bin/bash -c "pip install py-machineid==0.4.5 && python -c 'import machineid; print(machineid.id())'"

Thanks for the starting point, @juanbretti.

It works! Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants