Project not starting with custom username for web service

[taymless]

Is there an existing issue for this?

• I have searched the existing issues

Run a Diagnostic and Paste Link Here

Diagnostic runs fine, it's definitely something related to the project.

Expected Behavior

Starting my project with a custom docker-compose.environment.yaml file should just start.

Actual Behavior

ddev start fails with the following error messages:

host-username@macbook-pro core % ddev start
Starting project-name... 
Pushed mkcert rootca certs to ddev-global-cache/mkcert 
failed to solve: executor failed running [/bin/sh -c chmod 600 ~host-username/.pgpass ~host-username/.my.cnf]: exit code: 1 
Failed to start project-name: ComposeCmd failed to run 'COMPOSE_PROJECT_NAME=ddev-project-name docker-compose -f /Users/host-username/Developer/pec/core/.ddev/.ddev-docker-compose-full.yaml up --build -d', action='[up --build -d]', err='exit status 17', stdout='#1 [drud/ddev-dbserver-mysql-5.7:v1.21.2-project-name-built internal] load build definition from Dockerfile
#1 transferring dockerfile: 535B done
#1 DONE 0.0s

#2 [drud/ddev-webserver:v1.21.2-project-name-built internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.08kB done
#2 DONE 0.0s

#3 [drud/ddev-dbserver-mysql-5.7:v1.21.2-project-name-built internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [drud/ddev-webserver:v1.21.2-project-name-built internal] load .dockerignore
#4 transferring context: 2B done
#4 DONE 0.0s

#5 [drud/ddev-dbserver-mysql-5.7:v1.21.2-project-name-built internal] load metadata for docker.io/drud/ddev-dbserver-mysql-5.7:v1.21.2
#5 DONE 0.0s

#6 [drud/ddev-webserver:v1.21.2-project-name-built internal] load metadata for docker.io/drud/ddev-webserver:v1.21.2
#6 DONE 0.0s

#7 [drud/ddev-dbserver-mysql-5.7:v1.21.2-project-name-built 1/2] FROM docker.io/drud/ddev-dbserver-mysql-5.7:v1.21.2
#7 DONE 0.0s

#8 [drud/ddev-dbserver-mysql-5.7:v1.21.2-project-name-built 2/2] RUN (groupadd --gid 20 "host-username" || groupadd "host-username" || true) && (useradd  -l -m -s "/bin/bash" --gid "host-username" --comment '' --uid 501 "host-username" || useradd  -l -m -s "/bin/bash" --gid "host-username" --comment '' "host-username" || useradd  -l -m -s "/bin/bash" --gid "20" --comment '' "host-username")
#8 CACHED

#9 [drud/ddev-webserver:v1.21.2-project-name-built 1/5] FROM docker.io/drud/ddev-webserver:v1.21.2
#9 DONE 0.0s

#10 [drud/ddev-webserver:v1.21.2-project-name-built 2/5] RUN (groupadd --gid 20 "ec2-user" || groupadd "ec2-user" || true) && (useradd  -l -m -s "/bin/bash" --gid "ec2-user" --comment '' --uid 501 "ec2-user" || useradd  -l -m -s "/bin/bash" --gid "ec2-user" --comment '' "ec2-user" || useradd  -l -m -s "/bin/bash" --gid "20" --comment '' "ec2-user")
#10 CACHED

#11 [drud/ddev-webserver:v1.21.2-project-name-built 3/5] RUN apt-get -qq update && DEBIAN_FRONTEND=noninteractive apt-get -qq install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests cron
#11 CACHED

#12 [drud/ddev-webserver:v1.21.2-project-name-built 4/5] RUN export XDEBUG_MODE=off; composer self-update --stable || composer self-update --stable || true; composer self-update --2 || composer self-update --2 || true
#12 CACHED

#13 [drud/ddev-dbserver-mysql-5.7:v1.21.2-project-name-built] exporting to image
#13 exporting layers done
#13 writing image sha256:76f51ddff30208c678cee94e4d895cf13a890e97c187a56dc72c06dc78a2320e
#13 writing image sha256:76f51ddff30208c678cee94e4d895cf13a890e97c187a56dc72c06dc78a2320e done
#13 naming to docker.io/drud/ddev-dbserver-mysql-5.7:v1.21.2-project-name-built done
#13 DONE 0.0s

#14 [drud/ddev-webserver:v1.21.2-project-name-built 5/5] RUN chmod 600 ~host-username/.pgpass ~host-username/.my.cnf
#0 0.168 chmod: cannot access '~host-username/.pgpass': No such file or directory
#0 0.168 chmod: cannot access '~host-username/.my.cnf': No such file or directory
#14 ERROR: executor failed running [/bin/sh -c chmod 600 ~host-username/.pgpass ~host-username/.my.cnf]: exit code: 1
------
 > [drud/ddev-webserver:v1.21.2-project-name-built 5/5] RUN chmod 600 ~host-username/.pgpass ~host-username/.my.cnf:
#0 0.168 chmod: cannot access '~host-username/.pgpass': No such file or directory
#0 0.168 chmod: cannot access '~host-username/.my.cnf': No such file or directory
------
', stderr='failed to solve: executor failed running [/bin/sh -c chmod 600 ~host-username/.pgpass ~host-username/.my.cnf]: exit code: 1' 
host-username@macbook-pro core % 

Steps To Reproduce

I'm using the .ddev/docker-compose.environment.yaml file in my project to change the username for the web service.

  version: '3.6'

  services:
    web:
      build:
        args:
          username: ec2-user

This used to work just fine, until I upgraded to version 1.21.2 of ddev. It seems there is an issue with the generated file in .ddev/.webimageBuild/Dockerfile which looks like this:

#ddev-generated - Do not modify this file; your modifications will be overwritten.

### DDEV-injected base Dockerfile contents
ARG BASE_IMAGE
FROM $BASE_IMAGE

ARG username
ARG uid
ARG gid
RUN (groupadd --gid $gid "$username" || groupadd "$username" || true) && (useradd  -l -m -s "/bin/bash" --gid "$username" --comment '' --uid $uid "$username" || useradd  -l -m -s "/bin/bash" --gid "$username" --comment '' "$username" || useradd  -l -m -s "/bin/bash" --gid "$gid" --comment '' "$username")

### DDEV-injected from webimage_extra_packages or dbimage_extra_packages
RUN apt-get -qq update && DEBIAN_FRONTEND=noninteractive apt-get -qq install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests cron

### DDEV-injected composer update
RUN export XDEBUG_MODE=off; composer self-update --stable || composer self-update --stable || true; composer self-update --2 || composer self-update --2 || true

### DDEV-injected extra content

RUN chmod 600 ~host-username/.pgpass ~host-username/.my.cnf
ENV NVM_DIR=/home/host-username/.nvm

I believe the issue lies in the last 2 lines, which seems to use my host username instead of the $username.

Anything else?

No response

[rfay]

DDEV goes to great lengths to provide a web container with the username you use on your workstation. Can you say why you're trying to do this? I think it will likely be more involved than you think.

[taymless]

I've got the username for my infrastructure hard coded at a lot of places in my application. I'm sure that is the main issue here, but a rewrite is not my priority right now.

That is why I initially looked for a way to change the username in DDEV and was really happy with the solution of a custom yaml file. Like I said, the previous version worked fine with the environment.yaml

I didn't know the DDEV team was working hard at mapping the username to the web container. But still, shouldn't the auto generated file in .ddev/.webimageBuild/Dockerfile use the $username variable for all username related lines?

[rfay]

Can you say why it's important to change the username in the container?

[rfay]

I do see that the approach you're using used to work for solving complex usernames, https://stackoverflow.com/questions/64933879/umlaut-in-username-unicodedecodeerror-utf-8-codec-cant-decode-byte-0xe9-in

[taymless]

There are a few parts of my application that use the username ec2-user from my AWS infrastructure (cron jobs, file permissions, etc.). I'd obviously rather loose the username and have it work on any platform, but that's not going to happen right now.

I think I'll have to try and add a user during the DDEV initialisation, maybe as a hooks:post-start: exec command.

[rfay]

I was hoping to get you to explain why the username inside the container is important to you. I understand that "there are a few parts of your application" but hope you'll explain why the ddev-webserver user is important.

You could certainly add additional users in a .ddev/web-build/Dockerfile.* as well.

[taymless]

Sorry, didn't get that and wanted to avoid adding unnecessary details.

The "few parts" are actually mostly permission based issues that are all based on the user ec2-user. Like I said, that's coming from the AWS infrastructure we're using. The user/permissions issues we had were for cron jobs, dynamically added files/directories and jobs run by the webserver. So making the user for the ddev-webserver the same as on our production environment helped us to ensure that we can focus on issues that are not related to different environments/permissions but coming from application logic.

Hope this clears up the confusion.

Thanks for the hint, I'll check out the Dockerfile for adding the user I need.

[rfay]

I guess you have scripts running inside the web container that assume a specific user rather than using $USER?

Ok, please follow up with your fix here when you get it. I'll close this for now, as this isn't a normal use-case. Happy to continue the conversation or help out or whatever.

[rfay]

I'm working on this class of problem in https://github.com/rfay/ddev/tree/20221026_uid_collision , will be a PR soon. The "official" but off-topic issue is

• Error in web container on startup in Ubuntu Live session: failed to create symbolic link '/home/kubuntu/.nvm': Permission denied #4243

[rfay]

Please test with the artifacts at Please test with the artifacts at #4346 (comment)

[rfay]

You can now test with brew unlink ddev && brew install --HEAD--fetch-head ddev if you use homebrew, or get the binaries from https://github.com/drud/ddev/actions/runs/3364639330

[taymless]

Yeah, this fixed the issues I had. Thanks!