fix(webserver): better log-stderr.sh reporting, remove trixie-backports by stasadev · Pull Request #8042 · ddev/ddev

stasadev · 2026-01-20T11:06:20Z

The Issue

n-install.sh doesn't work if permissions on /usr/local/bin are changed:

cat > .ddev/web-build/Dockerfile.testing << 'EOF'
COPY --from=python:2.7-slim /usr/local /usr/local
EOF

ddev config --nodejs-version=18
ddev start

$ ddev exec node -v
v24.12.0

$ ddev logs
...
+ n-install.sh
This script cannot write to the directory /usr/local/bin
...

There are two problems here:

Permissions are reset because of COPY --from
There's no warning on ddev start about Node.js not being installed correctly.

I had this problem in:

fix: set permissions for /usr/local/bin ddev-python2#16

If people want to install libcurl4-openssl-dev in DDEV HEAD:

$ ddev exec 'sudo apt-get update && sudo apt-get install -y libcurl4-openssl-dev'
...
The following packages have unmet dependencies:
 libcurl4-openssl-dev : Depends: libcurl4t64 (= 8.14.1-2+deb13u2) but 8.18.0-1~bpo13+1 is to be installed
                        Depends: libnghttp3-dev (>= 1.1.0) but it is not going to be installed

The correct command will be:

ddev exec 'sudo apt-get update && sudo apt-get install libcurl4-openssl-dev/trixie-backports libnghttp3-dev/trixie-backports libngtcp2-dev/trixie-backports'

Or:

ddev exec 'sudo apt-get update && sudo apt-get install -t trixie-backports -y libcurl4-openssl-dev'

I don't see any benefit in using trixie-backports, because it's disabled by default, and you need to add */trixie-backports suffix for all dependencies or provide a -t trixie-backports flag. It turns out it's more complicated than I thought.

How This PR Solves The Issue

Replaces curl/trixie-backports with curl, because this change was going to break configuration for people who install libcurl-dev or any other dependencies on curl. I introduced this change in build(curl): use trixie-backports to install newer curl #7897, but it doesn't seem to be a good idea to use it.
Removes trixie-backports repository, people can install it manually if the want to, and this should be a part of the blog article Blog: All you need to know about installing software in DDEV ddev.com#383
Adds trixie-backports exclusively for the CiviCRM test, it doesn't fail locally, but always fails in GitHub Actions
Moves log-stderr.sh outside of n-install.sh, mariadb-client-install.sh, mysql-client-install.sh - any errors should be reported, not only network problems
Adds log-stderr.sh to mariadb-compat-install.sh
Fixes race condition with process substitution in log-stderr.sh by adding wait
Increases START_SCRIPT_TIMEOUT value, because we don't have install_nvm.sh anymore, see chore(deprecation): remove ddev nvm functionality in v1.25.0 #7826
Adds chmod -f ugo+rwx /usr/local/bin /usr/local/bin/* as one of the last commands in .ddev/.webimageBuild/Dockerfile

Manual Testing Instructions

Check permissions on /usr/local/bin:

cat > .ddev/web-build/Dockerfile.testing << 'EOF'
COPY --from=python:2.7-slim /usr/local /usr/local
EOF

ddev config --nodejs-version=18
ddev start

# see Node.js 18
ddev exec node -v

# should have correct permissions
ddev exec ls -ld /usr/local/bin

DDEV HEAD:

$ ddev exec node -v
v24.12.0

$ ddev exec ls -ld /usr/local/bin
drwxr-xr-x 1 root root 4096 Apr 20  2020 /usr/local/bin

This PR:

$ ddev exec node -v
v18.20.8

$ ddev exec ls -ld /usr/local/bin
drwxrwxrwx 1 root root 4096 Jan 20 12:51 /usr/local/bin

Test n-install.sh failure:

rm -f .ddev/web-build/Dockerfile.testing
ddev config --web-environment-add=HOSTNAME=""
ddev config --nodejs-version=14
ddev start
# see a warning on `ddev start`

DDEV HEAD:

$ ddev start
# no warning here, but it's in the logs:

$ ddev logs | grep -A2 log-stderr.sh
+ log-stderr.sh n-install.sh
This script requires HOSTNAME to be set
Command 'n-install.sh' didn't return stderr output

This PR:

$ ddev start
...
Warning: command 'n-install.sh' run as 'stas' failed with exit code 3:
This script requires HOSTNAME to be set
...

Test libcurl4-openssl-dev installation:

# Should be no errors here
ddev exec 'sudo apt-get update && sudo apt-get install -y libcurl4-openssl-dev'

DDEV HEAD:

$ ddev exec 'sudo apt-get update && sudo apt-get install -y libcurl4-openssl-dev'
...
The following packages have unmet dependencies:
 libcurl4-openssl-dev : Depends: libcurl4t64 (= 8.14.1-2+deb13u2) but 8.18.0-1~bpo13+1 is to be installed
                        Depends: libnghttp3-dev (>= 1.1.0) but it is not going to be installed
...

This PR:

$ ddev exec 'sudo apt-get update && sudo apt-get install -y libcurl4-openssl-dev'
# no errors, package should be installed

START_SCRIPT_TIMEOUT should be increased, because we don't have install_nvm.sh anymore:

ddev exec 'echo $START_SCRIPT_TIMEOUT'

DDEV HEAD, formula: (120 - 5) / 2 = 57

$ ddev exec 'echo $START_SCRIPT_TIMEOUT'
57

This PR, formula: 120 - 5 = 115

$ ddev exec 'echo $START_SCRIPT_TIMEOUT'
115

Automated Testing Overview

Release/Deployment Notes

github-actions · 2026-01-20T11:11:39Z

Download the artifacts for this pull request:

See Testing a PR.

tyler36

Does what the "manual testing" claims.

I tested with current stable v1.24.10 so some output did not match with expected "ddev head".
However, all "This PR" (v1.24.10-135-gb7dfb1079) output matched.

tyler36 · 2026-01-21T07:47:54Z

pkg/ddevapp/app_compose_template.yaml

-    # Currently used in 2 places: n-install.sh, install_nvm.sh
-    - START_SCRIPT_TIMEOUT={{ div (sub .DefaultContainerTimeout 5) 2 }}
+    # Currently used in 1 place: n-install.sh
+    - START_SCRIPT_TIMEOUT={{ div (sub .DefaultContainerTimeout 5) 1 }}


Suggested change

- START_SCRIPT_TIMEOUT={{ div (sub .DefaultContainerTimeout 5) 1 }}

- START_SCRIPT_TIMEOUT={{ sub .DefaultContainerTimeout 5 }}

What's the reasoning behind diving by 1 ((120 - 5) / 1 = 115)? It seems unnessasary from a maths point of view.

I left it here for better understanding what needs to be done, if we add some new extra script.

I can't remember syntax like this {{ div (sub .DefaultContainerTimeout X) Y }}

I'll add an example for it in the comments.

stasadev · 2026-01-21T11:00:01Z

docs/content/users/quickstart.md

 ```bash
 ddev exec "curl -LsS https://download.civicrm.org/latest/civicrm-STABLE-standalone.tar.gz -o /tmp/civicrm-standalone.tar.gz"
 ddev exec "tar --strip-components=1 -xzf /tmp/civicrm-standalone.tar.gz"
-ddev composer update civicrm/composer-compile-plugin --no-scripts


This is not needed anymore, because CiviCRM stable release already has the newer civicrm/composer-compile-plugin.

Not compatible with Composer 2.9 civicrm/composer-compile-plugin#32

rfay · 2026-01-21T15:41:09Z

In HEAD, I see the problem (nodejs version not set) but I don't see anything like this in ddev logs:

+ n-install.sh
This script cannot write to the directory /usr/local/bin

I doubt that matters, because this PR will fix the problem, but I didn't understand the OP completely apparently.

rfay

It's great, thank you!

Could you please add a whole usage guide to the top of log-stderr.sh ? It seems like there are correct ways to use it and different usages in different contexts.

rfay · 2026-01-20T16:41:53Z

containers/ddev-webserver/ddev-webserver-base-files/usr/local/bin/log-stderr.sh

 exit_code=$?

+# Wait for background process substitution (tee) to complete writing to file
+wait


WOW. I had no idea you could do this.

In HEAD, I see the problem (nodejs version not set) but I don't see anything like this in ddev logs:

+ n-install.sh This script cannot write to the directory /usr/local/bin

I doubt that matters, because this PR will fix the problem, but I didn't understand the OP completely apparently.

I think this is why you didn't see anything in the logs, because there was no wait.

rfay · 2026-01-21T15:51:49Z

containers/ddev-webserver/ddev-webserver-base-files/usr/local/bin/mariadb-client-install.sh

 # Run "apt-get update" manually only for mariadb and debian repos to make it faster
-log-stderr.sh apt-get update -o Acquire::Retries=5 -o Dir::Etc::sourcelist="sources.list.d/mariadb-archive.sources" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0" || exit $?
-log-stderr.sh apt-get update -o Acquire::Retries=5 -o Dir::Etc::sourcelist="sources.list.d/debian.sources" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0" || exit $?
+apt-get update -o Acquire::Retries=5 -o Dir::Etc::sourcelist="sources.list.d/mariadb-archive.sources" -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"


I didn't understand this change. Why does this no longer have log-stderr.sh ?

I moved log-stderr.sh outside the script:

if slices.Contains([]string{nodeps.MariaDB1011, nodeps.MariaDB114}, app.Database.Version) { extraWebContent = extraWebContent + "\nRUN log-stderr.sh mariadb-client-install.sh || true\n" }

stasadev · 2026-01-21T16:26:42Z

Could you please add a whole usage guide to the top of log-stderr.sh ? It seems like there are correct ways to use it and different usages in different contexts.

Added in 065f9a4 (this PR)

I didn't rebuild ddev-webserver for this comment change.

rfay · 2026-01-21T16:31:03Z

Thank you for adding usage, and there's no need for rebuild.

It does seem like there are two (or three?) different contexts for log-stderr.sh though

Image build
Additional layers in image build
start.sh or equivalent

Is that true? If so maybe the comment can include information about how those contexts should be used or managed differently.

stasadev · 2026-01-21T16:41:05Z

Updated it.

rfay · 2026-01-21T16:50:00Z

Thanks!

stasadev requested a review from a team as a code owner January 20, 2026 11:06

github-actions bot added bugfix dependencies Pull requests that update a dependency file labels Jan 20, 2026

stasadev changed the title ~~fix: better log-stderr.sh reporting, remove trixie-backports~~ fix(webserver): better log-stderr.sh reporting, remove trixie-backports Jan 20, 2026

stasadev force-pushed the 20260120_stasadev_stderr_logs branch from a004c7f to 505059e Compare January 20, 2026 11:48

stasadev requested a review from a team as a code owner January 20, 2026 13:57

tyler36 approved these changes Jan 21, 2026

View reviewed changes

stasadev added 2 commits January 21, 2026 12:47

fix: better log-stderr.sh reporting, remove trixie-backports

5ba493b

test(quickstart): use trixie-backports in civicrm test

38ccef5

stasadev force-pushed the 20260120_stasadev_stderr_logs branch from b7dfb10 to 82f735a Compare January 21, 2026 10:56

stasadev commented Jan 21, 2026

View reviewed changes

explain START_SCRIPT_TIMEOUT with example for N uses

8c73ee7

stasadev force-pushed the 20260120_stasadev_stderr_logs branch from 82f735a to 8c73ee7 Compare January 21, 2026 11:06

rfay approved these changes Jan 21, 2026

View reviewed changes

explain log-stderr.sh usage [skip ci]

065f9a4

make it clear about contexts [skip ci]

9c06c26

stasadev merged commit ee6c752 into ddev:main Jan 22, 2026
9 checks passed

stasadev deleted the 20260120_stasadev_stderr_logs branch January 22, 2026 07:42

chenrui333 mentioned this pull request Feb 3, 2026

ddev 1.25.0 chenrui333/homebrew-tap#4020

Closed

	- START_SCRIPT_TIMEOUT={{ div (sub .DefaultContainerTimeout 5) 1 }}
	- START_SCRIPT_TIMEOUT={{ sub .DefaultContainerTimeout 5 }}

Uh oh!

Conversation

stasadev commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The Issue

How This PR Solves The Issue

Manual Testing Instructions

Automated Testing Overview

Release/Deployment Notes

Uh oh!

github-actions bot commented Jan 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tyler36 left a comment

Choose a reason for hiding this comment

Uh oh!

tyler36 Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

stasadev Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

stasadev Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rfay commented Jan 21, 2026 • edited by stasadev Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rfay left a comment

Choose a reason for hiding this comment

Uh oh!

rfay Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

stasadev Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

rfay Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

stasadev Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

stasadev commented Jan 21, 2026

Uh oh!

rfay commented Jan 21, 2026

Uh oh!

stasadev commented Jan 21, 2026

Uh oh!

rfay commented Jan 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

stasadev commented Jan 20, 2026 •

edited

Loading

github-actions bot commented Jan 20, 2026 •

edited

Loading

stasadev Jan 21, 2026 •

edited

Loading

rfay commented Jan 21, 2026 •

edited by stasadev

Loading