-
Notifications
You must be signed in to change notification settings - Fork 15.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build: strip zip determinism on artifact upload #21756
build: strip zip determinism on artifact upload #21756
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we instead set the date to something deterministic when we create the zip file? e.g. the date from the commit?
It should be possible to set whichever date we choose, however all the existing tooling tend to assume the minimum epoch time settable in a zip as the default https://salsa.debian.org/reproducible-builds/strip-nondeterminism/blob/master/lib/File/StripNondeterminism/handlers/zip.pm#L200 . If we stick with that convention we will get free external tooling support for checking. |
@thypon in particular, I meant, can we set the date/time on the zip file when we create it, rather than after the fact? I think we could do so by setting |
Ah that should be the case, yep. My upload pass covers every case of zip file created, even when we don't directly control the zip creation. Moreover since it is located a step before uploading the release it also opens up for other future metadata removal (eg. OS version header or zero length trailers). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, ok, seems reasonable. Zip format probably won't change from underneath us I hope :)
@thypon Do you mind rebasing this PR on latest master? The macOS CI is failing because of an already fixed issue, rebasing on master should be able to turn it to green. |
Done, let's wait! |
Release builds running here: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like this change is erroring out with:
File "script\release\uploaders\upload.py", line 277, in <module>
sys.exit(main())
File "script\release\uploaders\upload.py", line 73, in main
upload_electron(release, electron_zip, args)
File "script\release\uploaders\upload.py", line 213, in upload_electron
_zero_zip_date_time(file_path)
File "script\release\uploaders\upload.py", line 175, in _zero_zip_date_time
archive_size = os.fstat(zip_.fileno()).st_size
AttributeError: 'str' object has no attribute 'fileno'
Command executed with exception: File "script\release\uploaders\upload.py", line 277, in <module>
sys.exit(main())
File "script\release\uploaders\upload.py", line 73, in main
upload_electron(release, electron_zip, args)
File "script\release\uploaders\upload.py", line 213, in upload_electron
_zero_zip_date_time(file_path)
File "script\release\uploaders\upload.py", line 175, in _zero_zip_date_time
archive_size = os.fstat(zip_.fileno()).st_size
AttributeError: 'str' object has no attribute 'fileno'
eg see https://ci.appveyor.com/project/electron-bot/electron-ia32-release/builds/30316079
Since electron zip are including build dates the checksum of each of these zip files is time dependant. In order to fix this issue strip all the dates contained in each of the zip entries.
Seems to be ok now |
Release builds running here: |
Merging as CI failures unrelated to PR change. |
Release Notes Persisted
|
Description of Change
Since electron zip are including build dates the checksum of each of
these zip files is time dependant. In order to fix this issue strip all
the dates contained in each of the zip entries.
See: #21327 and 97959b5 , cont #20949
Cc: @nornagon @felixrieseberg @ikkisoft
Checklist
Release Notes
Notes: Strip non-determinism from zip release files