Setting PYTHONDEVMODE environment variables causes seg fault

[dave-shawley]

Description of the issue

As a Python developer, I had the $PYTHONDEVMODE environment variable set in my shell to help me detect deprecations and resource leaks. I updated by docker installation and docker-compose started to fail due to the python memory debug support added in Python 3.6 (ref: [What's New in Python 3.6])(https://docs.python.org/3.6/whatsnew/3.6.html#pythonmalloc-environment-variable).

$
$
$ /usr/local/bin/docker-compose version
docker-compose version 1.28.5, build c4eb3a1f
docker-py version: 4.4.4
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020
$
$
$
$ PYTHONDEVMODE=1 /usr/local/bin/docker-compose version
Debug memory block at address p=0x7ff63d606690: API '�'
    15987178197214944733 bytes originally requested
    The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd):
        at p-7: 0xdd *** OUCH
        at p-6: 0xdd *** OUCH
        at p-5: 0xdd *** OUCH
        at p-4: 0x72 *** OUCH
        at p-3: 0x00 *** OUCH
        at p-2: 0x7c *** OUCH
        at p-1: 0x00 *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 8 pad bytes at tail=0xddde5dd41b3e446d are zsh: segmentation fault  PYTHONDEVMODE=1 /usr/local/bin/docker-compose version
$
$
$

Context information (for bug reports)

The following output is with the PYTHONDEVMODE environment variable unset.

Output of docker-compose version

docker-compose version 1.28.5, build c4eb3a1f
docker-py version: 4.4.4
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Output of docker version

Client: Docker Engine - Community
 Version:           20.10.5
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        55c4c88
 Built:             Tue Mar  2 20:13:00 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.5
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       363e9a8
  Built:            Tue Mar  2 20:15:47 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker-compose config

Not relevant since it happens regardless of the docker-compose configuration.

Steps to reproduce the issue

1. export PYTHONDEVMODE=1 in your shell
2. run docker-compose version

Observed result

Debug memory block at address p=0x7ff63d606690: API '�'
    15987178197214944733 bytes originally requested
    The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd):
        at p-7: 0xdd *** OUCH
        at p-6: 0xdd *** OUCH
        at p-5: 0xdd *** OUCH
        at p-4: 0x72 *** OUCH
        at p-3: 0x00 *** OUCH
        at p-2: 0x7c *** OUCH
        at p-1: 0x00 *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 8 pad bytes at tail=0xddde5dd41b3e446d are zsh: segmentation fault  PYTHONDEVMODE=1 /usr/local/bin/docker-compose version

Expected result

docker-compose version 1.28.5, build c4eb3a1f
docker-py version: 4.4.4
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Stacktrace / full error message

Debug memory block at address p=0x7ff63d606690: API '�'
    15987178197214944733 bytes originally requested
    The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd):
        at p-7: 0xdd *** OUCH
        at p-6: 0xdd *** OUCH
        at p-5: 0xdd *** OUCH
        at p-4: 0x72 *** OUCH
        at p-3: 0x00 *** OUCH
        at p-2: 0x7c *** OUCH
        at p-1: 0x00 *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 8 pad bytes at tail=0xddde5dd41b3e446d are zsh: segmentation fault  PYTHONDEVMODE=1 /usr/local/bin/docker-compose version

Additional information

OS version / distribution, docker-compose install method, etc.

• macOS Catalina, version 10.15.7
• MacBook Pro (15-inch, 2019)
• installed as part of docker desktop version 3.2.1
  • Engine: 20.10.5
  • Compose: 1.28.5
  • Kubernetes: 1.19.7
  • Notary: 0.6.1
  • Credential Helper: 0.6.3
  • Snyk:: 1.432.0

[ulyssessouza]

Hello @dave-shawley .

Since this is a memory allocation issue, I'm not sure if we can help with it. At most that could be a pyinstaller issue, but I'm not that certain of that.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it had not recent activity during the stale period.

[parfeniukink]

@dave-shawley Did you resolve the issue?

Try this:

export PYTHONDEVMODE=1
export PYTHONMALLOC=malloc

docker-compose version

sys:1: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020
compose/metrics/decorator.py:19: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/Library/Containers/com.docker.docker/Data/docker-cli.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback

And for docker-compose ps

$  docker-compose ps
sys:1: DeprecationWarning: PY_SSIZE_T_CLEAN will be required for '#' formats
Name   Command   State   Ports
------------------------------
compose/metrics/decorator.py:19: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/Library/Containers/com.docker.docker/Data/docker-cli.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
compose/cli/main.py:81: ResourceWarning: unclosed <socket.socket fd=5, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/.docker/run/docker.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
compose/cli/main.py:81: ResourceWarning: unclosed <socket.socket fd=6, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/.docker/run/docker.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
compose/cli/main.py:81: ResourceWarning: unclosed <socket.socket fd=7, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/.docker/run/docker.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
compose/cli/main.py:81: ResourceWarning: unclosed <socket.socket fd=8, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/.docker/run/docker.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
compose/cli/main.py:81: ResourceWarning: unclosed <socket.socket fd=9, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/.docker/run/docker.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
compose/cli/main.py:81: ResourceWarning: unclosed <socket.socket fd=10, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/.docker/run/docker.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
compose/cli/main.py:81: ResourceWarning: unclosed <socket.socket fd=11, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/dmytroparfeniuk/.docker/run/docker.sock>
ResourceWarning: Enable tracemalloc to get the object allocation traceback

[dave-shawley]

@parfeniukink i ended up wrapping /usr/local/bin/docker-compose in a shell script that unsets environment variables starting with PYTHON before exec’ing