Description
This issue is automatically created based on existing pull request: #39954: Fix concurrency issue in static content deploy
Summary
This fixes a bug in which multiple concurrent processes spin up to handle the same theme package, depending on how the themes are defined with their parents.
We saw a lot of issues like
DEBUG: STDERR:Compilation from source /path/to/some.js failed
Magento\Framework\Exception\FileSystemException: The file or directory "/app/var/view_preprocessed/pub/static/frontend/path/to/some.js" cannot be copied to "/app/pub/static/frontend/path/to/some.js"
We also saw less compilation intermittently failing to generate complete styles-m.css and styles-l.css files, as multiple processes were fighting and doing work on the same file at the same time. This cause the frontend to be deployed in a broken manner which needed a redeploy to solve.
For affected setups this may also improve the performance of this process, by the idea that we have a process no longer doing duplicated work.
To reproduce
Reproduce
Clone https://github.com/convenient/magento-bug-reproduction-static-content-concurrency then run composer install, you can now generate static content.
Without the bugfix we can see multiple executions of the same package, see that frontend/Custom/ztheme/default is executed multiple times
$ rm -rf pub/static var/view_preprocessed/ var/cache ; php bin/magento > /dev/null ; php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB en_IE -vvv | grep -E 'Execute: frontend/Custom/ztheme|Prevent'
Execute: frontend/Custom/ztheme/default
Execute: frontend/Custom/ztheme/default
Execute: frontend/Custom/ztheme/en_US
Execute: frontend/Custom/ztheme/en_GB
Execute: frontend/Custom/ztheme/en_IEWith the bugfix we can see each is only executed once
$ rm -rf pub/static var/view_preprocessed/ var/cache ; php bin/magento > /dev/null ; php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB en_IE -vvv | grep -E 'Execute: frontend/Custom/ztheme|Prevent'
Execute: frontend/Custom/ztheme/default
Preventing duplicate execution of package as it is in progress: frontend/Custom/ztheme/default (pid: 67830)
Execute: frontend/Custom/ztheme/en_US
Execute: frontend/Custom/ztheme/en_GB
Execute: frontend/Custom/ztheme/en_IEVerification
I have used this fix on production for over a year without any issues. But we can also double check on the example reproduction repository provided above.
Get a baseline
rm -rf pub/static/
php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB
find ./pub/static -type f -exec md5sum {} + | awk '{print $1, $2}' > before.txtApply the bugfix then do the following
rm -rf pub/static/
php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB
find ./pub/static -type f -exec md5sum {} + | awk '{print $1, $2}' > after.txtSee that all files (that matter) were generated the same
$ diff before.txt after.txt
1c1
< 04d5dfe8b06b35834eeb526128ef11ba ./pub/static/deployed_version.txt
---
> b5e5f1122730b93197d32286a8dc65ab ./pub/static/deployed_version.txtContribution checklist (*)
- Pull request has a meaningful description of its purposeAll commits are accompanied by meaningful commit messagesAll new or changed code is covered with unit/integration tests (if applicable)README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an updateAll automated tests passed successfully (all builds are green)
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Milestone
Relationships
Participants
Activity
engcom-Dash commentedon Jun 12, 2025
Hello @convenient Thank you for report & collaboration!
We have tried to reproduce the issue in the latest development branch i.e. 2.4-develop and the issue has been reproducible for us.
Several concurrent processes get initiated to manage the same theme package, depending on how the themes are structured and their relationships with parent theme is defined.
Please refer to the below screenshots for reference:
rm -rf pub/static var/view_preprocessed/ var/cache ; php bin/magento > /dev/null ; php ./bin/magento setup:static-content:deploy --no-ansi --no-interaction -f -s compact -vvv --jobs 4 --no-html-minify en_US en_GB en_IE -vvv | grep -E 'Execute: frontend/Magento/blank|Prevent'
Hence, confirming the issue.
m2-assistant commentedon Jun 12, 2025
Hi @engcom-November. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇
Area: XXXXXlabel to the ticket, indicating the functional areas it may be related to.2.4-developbranchDetails
- If the issue is reproducible on2.4-developbranch, please, add the labelReproduced on 2.4.x.- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
Issue: Confirmedonce verification is complete.github-jira-sync-bot commentedon Jun 12, 2025
✅ Jira issue https://jira.corp.adobe.com/browse/AC-14944 is successfully created for this GitHub issue.
m2-assistant commentedon Jun 12, 2025
✅ Confirmed by @engcom-Dash. Thank you for verifying the issue.
Issue Available: @engcom-Dash, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.