fix: enforce LF line endings for shell scripts, gradlew, KEYS#15620
Closed
jamesfredley wants to merge 1 commit into8.0.xfrom
Closed
fix: enforce LF line endings for shell scripts, gradlew, KEYS#15620jamesfredley wants to merge 1 commit into8.0.xfrom
jamesfredley wants to merge 1 commit into8.0.xfrom
Conversation
The release verification flow runs in a Linux container built from etc/bin/Dockerfile, but the repository had no .gitattributes. With Git for Windows' default core.autocrlf=true, every Windows checkout converts gradlew, etc/bin/*.sh, the Dockerfile, and the KEYS file from LF to CRLF in the working tree. The Dockerfile's ADD then preserves those CRLFs into the image, with three failure modes: 1. The Linux kernel refuses to execute scripts with `bash\r` in the shebang, so gradlew and every verify-*.sh script fails with "cannot execute: required file not found". 2. verify-keys.sh hashes the in-tree KEYS file and compares against https://dist.apache.org/repos/dist/release/grails/KEYS, which is stored as LF. The hashes mismatch even though the content is byte-identical after line-ending normalization. 3. By extension, verify.sh aborts at its first step on any Windows committer's machine. Fix at the right layer (git's working-tree write path): - Add .gitattributes pinning *.sh, gradlew, KEYS, Dockerfile, and *.properties to eol=lf, while keeping *.bat / *.cmd / gradlew.bat as eol=crlf and marking common binary types as binary so git never tries to normalize them. Default `* text=auto` lets git auto-detect everything else. - Add a defensive sed-strip step in etc/bin/Dockerfile after the ADD lines, so even if a committer feeds a CRLF-corrupted build context (existing local checkout under autocrlf=true), the image still ends up with LF scripts. Existing index content is unchanged: every affected file is already stored as LF in the index. This commit only changes what `git checkout` writes to the working tree on Windows and adds belt-and- braces normalization at image-build time. CI on ubuntu-latest is unaffected. Linux/macOS verifiers are unaffected. Windows verifiers can now run the container build flow documented in RELEASE.md without manual workarounds. Assisted-by: claude-code:claude-opus-4-7
✅ All tests passed ✅🏷️ Commit: 0cc3558 Learn more about TestLens at testlens.app. |
jdaugherty
approved these changes
May 2, 2026
| # still feed CRLF into the build context. Linux refuses scripts with bash\r | ||
| # shebangs and verify-keys.sh would fail the byte-for-byte SVN comparison. | ||
| # Strip CRs from any text file that must be LF on Linux. | ||
| RUN find /home/groovy/scripts -type f \( -name '*.sh' -o -name 'gradlew' -o -name 'KEYS' -o -name '*.properties' \) -exec sed -i 's/\r$//' {} \; |
Contributor
There was a problem hiding this comment.
So this is really only for windows? I have never had this problem on a mac
Contributor
Author
There was a problem hiding this comment.
100%, I had fixed it a long time ago, but now have 4 copies of grails-core across a few machines and ran into it again.
|
The change adds defensive line-ending normalization in the Dockerfile to strip carriage returns (CR) from scripts and related files, addressing potential CRLF issues from Windows users despite .gitattributes settings. Macs use LF by default, so this problem is less common there, but the fix ensures compatibility across platforms. |
Contributor
Author
|
Replaced by #15622 so we have this change on 7.0.x -> 8.0.x |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pin the line endings of shell scripts,
gradlew,KEYS, and the Dockerfile to LF on every platform, and add a defensivesedstrip step inetc/bin/Dockerfile, so the release verification container is reproducible from a Windows committer's checkout.Why
There is no
.gitattributesin the repo today. With Git for Windows' defaultcore.autocrlf=true, every Windows checkout converts these files from LF to CRLF in the working tree. The Dockerfile'sADDthen preserves the CRLF into the image, which produces three independent failures during release verification:bash\rin the shebang, sogradlewand everyverify-*.shscript fail withcannot execute: required file not found//usr/bin/env: 'bash\r': No such file or directory.verify-keys.shshasum-512s the in-treeKEYSfile and compares against the canonical copy athttps://dist.apache.org/repos/dist/release/grails/KEYS. The two byte-streams differ by exactly one\rper line (52 bytes total) and the comparison fails even though the content is byte-identical after normalization.verify.shaborts at its first step (verify-keys.sh) on any Windows committer's machine. The container documented inRELEASE.mdis unusable as-is.CI on
ubuntu-latestand any Linux/macOS verifier are unaffected. The bug is only visible to a Windows committer because git's checkout-time conversion is the layer that introduces the corruption.Why fix this at the
.gitattributeslayer.gitattributesis the only mechanism in the repo that operates at git's working-tree write path. Other code-style mechanisms either don't run at the right time or don't cover the right files:*.sh/gradlew/KEYS?.editorconfig(existing - groovy/java only)codeStyletask.gitattributes(this PR)Changes
.gitattributes(new)* text=auto- sane default; git auto-detects text vs binary, stores text as LF in the index*.sh,gradlew,KEYS,Dockerfile,*.properties->text eol=lf(must be LF on Linux verifier and CI)*.bat,*.cmd,gradlew.bat->text eol=crlf(must be CRLF on Windows)*.jar,*.zip,*.png,*.pdf,*.gpg,*.keystore, ...) markedbinaryso git never normalizes themetc/bin/DockerfileAdd a
RUN sed -i 's/\r$//'step after theADDlines, scoped to*.sh,gradlew,KEYS, and*.properties. This is belt-and-braces: even if a committer's working tree still has CRLF (because their local checkout predates this PR and they have not re-checked-out / renormalized), the image they build will still be correct.Index impact
Verified ahead of time:
Every affected file is already LF in the index. Only the working tree is CRLF on Windows. This PR therefore does not require
git add --renormalize .and does not produce any blob churn - it only changes whatgit checkoutwrites to disk on the next checkout.Local verification
Done on a Windows machine with
core.autocrlf=true:docker build -f etc/bin/Dockerfile .succeeds, but inside the containergradlew --version,verify.sh v..., andverify-keys.shall fail with thebash\r/ hash-mismatch errors described above.RUN sedstep alone, since no re-checkout was needed locally): the image'sgradlew --versionsucceeds and downloads Gradle 9.4.1,verify.shreaches its arg-validation step, andverify-keys.shpasses the byte-for-byte SVN comparison againstdist.apache.org/repos/dist/release/grails/KEYS.Related
Companion to #15619 (
docs: update RELEASE.md examples for 8.0.0-M1). With both PRs merged, the container verification flow on8.0.xis ready for the upcoming 8.0.0-M1 milestone release on every platform.