Make the build reproducible #5019
Conversation
Whilst working on the Reproducible Builds effort [0] I noticed that scrapy could not be built reproducibly. This is due to the documentation embedding the current build year in the generated files, therefore making the build vary depending on when you build it. The fix is to use SOURCE_DATE_EPOCH [1] if it is exported to the surrounding environmnent. I originally filed this in Debian as bug #983852 [2]. [0] https://reproducible-builds.org/ [1] https://reproducible-builds.org/specs/source-date-epoch/ [2] https://bugs.debian.org/983852
|
Auto-updating copyright at build time is a bad practice anyway, because building today's software next year, would produce so the better fix could be to state |
|
I agree that it’s a bad practice, only better than not updating it at all. The best I can think of at the moment is to go for a pre-commit hook that updates the year if it is out of date. But then you need to get contributors to install pre-commit. Alternatively, we could use c4urself/bump2version#133 once implemented to make sure we update the year before every release. |
|
Closing and reopening to re-run the tests with the latest changes from |
Codecov Report
@@ Coverage Diff @@
## master #5019 +/- ##
==========================================
- Coverage 88.01% 87.65% -0.36%
==========================================
Files 158 162 +4
Lines 9726 10301 +575
Branches 1433 1501 +68
==========================================
+ Hits 8560 9029 +469
- Misses 911 1000 +89
- Partials 255 272 +17
|
Co-authored-by: Adrián Chaves <adrian@chaves.io>
| build_date = datetime.utcfromtimestamp( | ||
| int(os.environ.get('SOURCE_DATE_EPOCH', time.time())) | ||
| ) | ||
| copyright = f'2008–{build_date.year}, Scrapy developers' |
There was a problem hiding this comment.
| build_date = datetime.utcfromtimestamp( | |
| int(os.environ.get('SOURCE_DATE_EPOCH', time.time())) | |
| ) | |
| copyright = f'2008–{build_date.year}, Scrapy developers' | |
| copyright = f'2008–2023, Scrapy developers' |
... plus removing the datetime, os, and time imports from this configuration module.
(rationale: it's nice to automate where possible -- and authorship doesn't expire -- but copyright does, and figuring that out (and updating notices like this) usually benefits from human thought and discussion)
Whilst working on the Reproducible Builds effort I noticed that scrapy could not be built reproducibly
This is due to the documentation embedding the current build year in the generated files, therefore making the build vary depending on when you build it. The fix is to use
SOURCE_DATE_EPOCHif it is exported to the surrounding environment.I originally filed this in Debian as bug #983852.