-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Reproducible Builds] Diff in generated/metadata between the builds #23324
Comments
Hi @ihor-sviziev. Thank you for your report.
Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:
For more details, please, review the Magento Contributor Assistant documentation. @ihor-sviziev do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?
|
Please explain.
This is perfectly valid. However, system behavior must not vary depending on order of generation. https://reproducible-builds.org/ has nothing to do with this issue: it's all about no backdoors in compilers. |
Hi @orlangur,
Result files after DI compile should be exactly the same when you running it from one commit with the same software versions, in my case - single docker image (including image hash) was used in all cases. But result is different.
It might not vary right now, but at some point it will because of some bug will be introduced. This part isn’t covered by 100% with tests.
Directly - yes, it’s not related, but if we’ll look from another angle - it’s related. you can see example from symfony framework- they’re using the same approach https://symfony.com/blog/new-in-symfony-reproducible-builds
For magento we could say the same - if you run DI compile of the same unchanged application multiple times, the result should be the same. |
Just provide the difference. No buzz :) |
Diff really huge, but just to understand - in build 1 we have
In build 2 we have following:
All classes are present in both files, no differences between the data, just sort order is different. I just showed for 1 custom module, but it appears for all modules, including Magento core. |
@ihor-sviziev I see, just some class list in order of collecting. Did it ever cause any issue? My understanding is that we need to fix underlying bugs in case order matters in some cases but generally list sorting each time looks like an overkill and can slowdown significantly as well. Is there a way to put files and directories in pre-determined order into filesystem maybe? |
I’m not sure if it ever created issues there, but you even can’t test it because result even on single machine might be different now, you can’t say that this version is correct and this is not. There actually similar issues with static content deploy - you’re getting different content of files js-translation.json and different content of js bundle files, but I still didn’t get to it. |
Sorry, my bad, if we just sort paths and not full list of files sorting may be negligible. Will double check.
I meant real bugs caused by nondeterministic order of items in list.
Does Symfony enforce SB with some build? |
I didn't saw any real bugs caused by this issue. I think just because this issue is not persistent.
I reviewed all PRs linked to symfony/symfony#25958 - found only one check - https://github.com/symfony/symfony/pull/26176/files#diff-f410485b2fdabe90d3c911496fbe9ea9R64 Actually test also could be like this: |
Hi @orlangur, |
@ihor-sviziev sorry for delay, will post an update this week. |
Hi @engcom-Charlie. Thank you for working on this issue.
|
✅ Confirmed by @engcom-Charlie Issue Available: @engcom-Charlie, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself. |
Hi @ihor-sviziev. Thank you for working on this issue.
|
Hi @ihor-sviziev. Thank you for your report. The fix will be available with the upcoming 2.3.4 release. |
Hi @ihor-sviziev. Thank you for your report. The fix will be available with the upcoming 2.4.4 release. |
It's really important to have Reproducible Builds. It will help to check diff between the builds, so you'll be sure that you deploy only needed changes to production.
Preconditions (*)
Steps to reproduce (*)
php bin/magento setup:di:compile
diff -rq ./build-1/ ./build-2/
Expected result (*)
Actual result (*)
Additional info
This issue reproducing all the time on Bitbucket Pipelines, but locally I can't reproduce it. I think it caused by using different servers on each build on CI.
If we'll try to compare these files after sorting - they actually the same. I sorted them using following script:
The text was updated successfully, but these errors were encountered: