Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please support SOURCE_DATE_EPOCH specification for build time stamps #938

Closed
brianmay opened this issue May 15, 2016 · 8 comments
Closed

Please support SOURCE_DATE_EPOCH specification for build time stamps #938

brianmay opened this issue May 15, 2016 · 8 comments
Labels
Bug
Milestone

Comments

@brianmay
Copy link

@brianmay brianmay commented May 15, 2016

Copied from http://bugs.debian.org/824266:

Hi,

mkdocs integrates build time stamps into the documentation it is
generating. This makes at least unburden-home-dir no more reproducibly
building because it now uses mkdocs to generate HTML documentation at
build time.

See https://tests.reproducible-builds.org/rb-pkg/unstable/amd64/unburden-home-dir.html and
https://tests.reproducible-builds.org/dbd/unstable/amd64/unburden-home-dir_0.4.diffoscope.html

Other cases of this issue are the source package djangorestframework (see
https://tests.reproducible-builds.org/rb-pkg/testing/i386/djangorestframework.html and
https://tests.reproducible-builds.org/dbd/testing/i386/djangorestframework_3.3.2-2.diffoscope.html)
and of course the source package python-mkdocs itself, see
https://tests.reproducible-builds.org/rb-pkg/unstable/amd64/python-mkdocs.html and
https://tests.reproducible-builds.org/dbd/unstable/amd64/python-mkdocs_0.15.3-3.diffoscope.html

At https://reproducible-builds.org/specs/source-date-epoch/ you can find
a bound to be widely adopted specification for how to solve such kind of
issues generally: If the environment variable SOURCE_DATE_EPOCH exists
use the Unix time stamp (seconds since 1st of January 1970, 00:00:00
UTC) it contains to preseed all build time stamps with that time stamp
instead of the current time.

See https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal#Examples
for some examples on how to implement support for SOURCE_DATE_EPOCH,
including an example for Python.

@d0ugal d0ugal added the Bug label May 16, 2016
@d0ugal
Copy link
Member

@d0ugal d0ugal commented May 16, 2016

Yup, this change seems reasonable. Thanks for the report.

@d0ugal d0ugal added this to the 0.16 milestone May 16, 2016
@waylan
Copy link
Member

@waylan waylan commented May 16, 2016

See https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal#Examples
for some examples on how to implement support for SOURCE_DATE_EPOCH,
including an example for Python.

FYI, the Python example is wrong. If SOURCE_DATE_EPOCH is supposed to be a Unix timestamp (number of seconds since epoch), then time.gmtime() is not the Python equivalent. However, calendar.timegm(datetime.datetime.utcnow().utctimetuple()) or calendar.timegm(time.gmtime()) is. It is annoyingly complicated to get a Unix timestamp in Python. Although Python >= 3.3 makes it easier with datetime.datetime.utcnow().timestamp(). Not sure how the datetime module went so long without the ability to return a timestamp.

@mapreri
Copy link

@mapreri mapreri commented May 20, 2016

FYI, the Python example is wrong. If SOURCE_DATE_EPOCH is supposed to be a Unix timestamp (number of seconds since epoch), then time.gmtime() is not the Python equivalent. However, calendar.timegm(datetime.datetime.utcnow().utctimetuple()) or calendar.timegm(time.gmtime()) is. It is annoyingly complicated to get a Unix timestamp in Python. Although Python >= 3.3 makes it easier with datetime.datetime.utcnow().timestamp(). Not sure how the datetime module went so long without the ability to return a timestamp.

meh, yeah time.gmtime() returns a time.struct_time instance.
umh, and what about time.time()? From https://docs.python.org/3.5/library/time.html :

Return the time in seconds since the epoch as a floating point number. Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls.

should be exactly what is needed (maybe wrapped in int())?

@infinity0
Copy link

@infinity0 infinity0 commented May 23, 2016

Thanks for catching that. Yes we did originally have time.time() which is simplest but somehow apparently I managed to fuck it up here possibly by hitting ctrl-z or something at the wrong time. I don't remember adding the gm but I do remember doing the rest of that edit.

@brianmay
Copy link
Author

@brianmay brianmay commented Jul 23, 2016

Hello,

I have been told this this change by itself is not sufficient for reproducible builds.

See https://bugs.debian.org/831648

Looks like a similar change is required for mkdocs/nav.py,in particular the update_time variable.

--- python-mkdocs-0.15.3.orig/mkdocs/nav.py
+++ python-mkdocs-0.15.3/mkdocs/nav.py
@@ -137,7 +137,11 @@ class Page(object):
         self.abs_url = url
         self.active = False
         self.url_context = url_context
-        self.update_date = datetime.datetime.now().strftime("%Y-%m-%d")
+
+        try:
+            self.update_date = datetime.datetime.utcfromtimestamp(int(os.environ['SOURCE_DATE_EPOCH']))
+        except KeyError:
+            self.update_date = datetime.datetime.now().strftime("%Y-%m-%d")

         # Relative paths to the input markdown file and output html file.
         self.input_path = path
@waylan
Copy link
Member

@waylan waylan commented Jul 24, 2016

Ah, the time stamp for each individual page needs to honor the env variable also.

@waylan waylan reopened this Jul 26, 2016
waylan added a commit to waylan/mkdocs that referenced this issue Aug 4, 2016
@waylan waylan closed this in #1010 Aug 5, 2016
waylan added a commit that referenced this issue Aug 5, 2016
@DaniellMesquita
Copy link

@DaniellMesquita DaniellMesquita commented Jan 5, 2020

MkDocs builds are now reproducible?

@waylan
Copy link
Member

@waylan waylan commented Jan 6, 2020

Yes, since version 0.17.0 (released 2017-10-19) a build is reproducible so long as the SOURCE_DATE_EPOCH environment variable is set to the same value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.