/proc/meminfo no longer reports Hugepagesize in 4.26.1

[joelrbrandt]

Description

Starting with Docker Desktop for Mac 4.26.1, /proc/meminfo no longer reports Hugepagesize.

I have only tested versions 4.25.2 and 4.26.1. It is reported in 4.25.2, and is not in 4.26.1

This value is read by oneTBB here, and when it can't be read, it results in this assert:

Huge Page size can't be zero if we found thp existence.

Output from an ubuntu:latest container in 4.25.2 (correct output):

$ docker run --rm ubuntu cat /proc/meminfo
MemTotal:       32821432 kB
MemFree:        24376688 kB
MemAvailable:   30888924 kB
Buffers:          296140 kB
Cached:          6061964 kB
SwapCached:            0 kB
Active:          1961464 kB
Inactive:        5663580 kB
Active(anon):       3308 kB
Inactive(anon):  1267224 kB
Active(file):    1958156 kB
Inactive(file):  4396356 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       1048572 kB
SwapFree:        1048572 kB
Dirty:                76 kB
Writeback:             0 kB
AnonPages:       1242464 kB
Mapped:           328260 kB
Shmem:              3592 kB
KReclaimable:     543476 kB
Slab:             621132 kB
SReclaimable:     543476 kB
SUnreclaim:        77656 kB
KernelStack:       11716 kB
PageTables:        25804 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    17459288 kB
Committed_AS:    3352592 kB
VmallocTotal:   133143592960 kB
VmallocUsed:       16116 kB
VmallocChunk:          0 kB
Percpu:             3800 kB
AnonHugePages:    595968 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB

Output from an ubuntu:latest container in 4.26.1 (problematic output):

$ docker run --rm ubuntu cat /proc/meminfo
MemTotal:       32823420 kB
MemFree:        31190868 kB
MemAvailable:   31677968 kB
Buffers:          224252 kB
Cached:           491480 kB
SwapCached:            0 kB
Active:           328900 kB
Inactive:         801764 kB
Active(anon):        884 kB
Inactive(anon):   416168 kB
Active(file):     328016 kB
Inactive(file):   385596 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       1048572 kB
SwapFree:        1048572 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:        412432 kB
Mapped:           272640 kB
Shmem:              1192 kB
KReclaimable:     294424 kB
Slab:             334332 kB
SReclaimable:     294424 kB
SUnreclaim:        39908 kB
KernelStack:        9384 kB
PageTables:         4812 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    17460280 kB
Committed_AS:    2473532 kB
VmallocTotal:   133141626880 kB
VmallocUsed:       13680 kB
VmallocChunk:          0 kB
Percpu:             2600 kB
AnonHugePages:    327680 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB

Reproduce

1. run docker run --rm ubuntu cat /proc/meminfo
2. look for a Hugepagesize entry in the output

Expected behavior

There should be a Hugepagesize entry

docker version

Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:04:20 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.26.1 (131620)
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:08:15 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    24.0.7
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.12.0-desktop.2
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.23.3-desktop.2
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.21
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  0.1
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.10
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-scan
  scout: Docker Scout (Docker Inc.)
    Version:  v1.2.0
    Path:     /Users/jbrandt/.docker/cli-plugins/docker-scout

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 127
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.5.11-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 31.3GiB
 Name: docker-desktop
 ID: 584c4311-67a0-4402-95a8-3b939b3b174d
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

CB57EE1C-B6BA-4A90-8B85-A1525694B3FF/20231216175834

Additional Info

No response

[dgageot]

Duplicate of #7115

[dgageot]

Actually, when we removed Huge Pages, we probably forgot to also remove Transparent Huge Pages. Until we find how to re-enable HP, I'll remove THP, this should fix the inconsistency you see, @joelrbrandt.
Do you have a docker command I can use to reproduce your use case?

[joelrbrandt]

@dgageot thanks for looking into this!

Apologies, but I'm not sure what you're looking for in repro steps beyond what I put in the issue description.

The issue we're having is that oneTBB throws this assert when it tries to allocate memory:
https://github.com/oneapi-src/oneTBB/blob/3b9f9baef9c27e4d22ebeff59118aaddaa40e9f2/src/tbbmalloc/tbbmalloc_internal.h#L488

The reason it throws that assert is because:

1. it checks for the existence of THP by doing this: https://github.com/oneapi-src/oneTBB/blob/3b9f9baef9c27e4d22ebeff59118aaddaa40e9f2/src/tbbmalloc/tbbmalloc_internal.h#L482-L485
2. it checks for the size of huge pages by doing this: https://github.com/oneapi-src/oneTBB/blob/3b9f9baef9c27e4d22ebeff59118aaddaa40e9f2/src/tbbmalloc/tbbmalloc_internal.h#L456-L463

It gets a y for the first check, and no huge page size for the second check, so the assert fires.

I guess the two Docker commands to demonstrate this behavior would be:

1. docker run --rm ubuntu cat /sys/kernel/mm/transparent_hugepage/enabled | grep "\[always\]", which produces a match
2. docker run --rm ubuntu cat /proc/meminfo | grep Hugepagesize, which does not produce a match

Anyway, your theory about removing THP should indeed fix this inconsistency (assuming the first command above no longer produces a grep match).

LMK if I can help further!

[dgageot]

Anyway, your theory about removing THP should indeed fix this inconsistency (assuming the first command above no longer produces a grep match).

Thanks a lot for all the details!
Yes, I think it will fix it.
To be 100% sure, I was looking for a command that runs your workflow, to check that it's really fixed. I would expect some Dockerfile or piece of code that uses openTBB and that currently fails for you.

[dgageot]

Hi everyone! Turns out we found how to re-enable Huge Pages, keep Transparent Huge Pages and still fix the php issue. This should ship in Docker Desktop 4.27.

[joelrbrandt]

Woohoo! Thanks for the work on this, and the update, @dgageot !

[dgageot]

@joelrbrandt Could you test Docker Desktop 4.27 and close this issue if it's fixed?

[joelrbrandt]

@dgageot confirming that I was able to repro the problem in 4.26.1 using our latest code (I hadn't tried in awhile), and that I was not able to repro on 4.27.0. It's fixed 🙌

Thank you for your attention to this issue, and for closing the loop when it was fixed! Closing this issue!