Bad parsing of hashes in env files

[nuno-andre]

Description
Bad parsing of hashes in env files.

Steps to reproduce the issue:

1. Create an .env file

   TEST=123#4

2. Create a compose.yaml

   services:
     test:
       image: alpine
       env_file: .env
       command: tail -f /dev/null
       container_name: test

3. Run

   docker-compose up
   docker exec test env

Describe the results you received:

TEST=123

Describe the results you expected:

TEST=123#4

E.g.:

$ docker exec test sh -c 'TEST=123#4; printf "$TEST"'
123#4

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker compose version:

Docker Compose version v2.5.1

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)
  compose: Docker Compose (Docker Inc., v2.5.1)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 7
  Running: 1
  Paused: 0
  Stopped: 6
 Images: 55
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.0.3-0-gf46b6ba
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.60.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 5.73GiB
 Name: docker-desktop
 ID: 2DPX:3CK6:H2AL:77NJ:PPWO:LP77:FRFU:7LOO:K4NQ:ALRU:KGCA:HEXJ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional environment details:

[inFocus7]

I tested this locally and it worked as-expected on Compose v2.2.3 (Desktop v4.5, Engine 20.10.12). Then I updated my Docker Desktop to the latest (Desktop v4.8.2), and now on Compose v2.5.1, where its broken needing quotation marks. So something must've changed between Feb to now. :rip:

I'm going to look more into it right now and I'll update/create a PR if I find the core problem 👀

edit: put update in below comment.

[inFocus7]

Update
Heyyo!

This is the expected parsing of that .env value. You'll want to wrap the value in quotes if you want to use # as part of the value, else it's seen as an inline comment and it gets ignored.

So instead of TEST=123#4, you'll want to do TEST="123#4"

---

This change was introduced in compose-go v1.10 in this PR, used in Docker Compose v2.3.0+.

[nuno-andre]

Hi @inFocus7!

Thanks for pointing out the commit.

IMHO, this is a parsing error. env files come from *nix and, to my knowledge, no shell processes the strings in this way. An inline comment not only needs to start with a hash, but also with a whitespace in front of it.

TEST1=123 # inline comment
TEST2=123#not-an-inline-comment
TEST3="123 # not a comment either"

GitHub's syntax highlighter agrees with this ;)

[prullmann]

Thanks @nuno-andre for reporting this issue!
You helped me find out why one of my server passwords didn't work as expected.

[ShadowLNC]

Since this is the top result on Google (for my search), adding my findings here:

According to the release notes, compose-go v1.2.9 includes the linked PR, and according to Docker Compose release notes, Compose v2.7.0 changes the compose-go version to v1.2.9 in this PR.

Somewhere along the way, I think this regressed - I was still seeing the issue (Compose 2.10.2 as part of Docker Desktop for Mac 4.12.0).

I did some more searching - another PR was released in compose-go v1.6.0, which in turn is included in Compose v2.11.2.

More details about the fix are in issue #9879 (Compose repo). Docker Desktop updates approximately monthly (source - link anchor scrolls too far) and includes the newest Compose version (source).