Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PATCH: Honour SOURCE_DATE_EPOCH environment variable for reproducible output (Origin: bugzilla #751984) #5826

Closed
doxygen opened this Issue Jul 2, 2018 · 0 comments

Comments

Projects
None yet
1 participant
@doxygen
Copy link
Owner

doxygen commented Jul 2, 2018

status RESOLVED severity enhancement in component general for ---
Reported in version 1.8.11-GIT on platform Other
Assigned to: Dimitri van Heesch

Original attachment names and IDs:

On 2015-07-05 13:06:59 +0000, marivalenm@gmail.com wrote:

Created attachment 306856
Honour SOURCE_DATE_EPOCH environment variable for reproducible output

In Debian there are 178 source packages that are not byte for byte reproducible between different rebuilds because doxygen includes a timestamp in its HTML, LaTeX, and Man page output [1] and [2].

Many packages became immediately reproducible after Doxygen changed its HTML_TIMESTAMP default from YES to NO but this still left many packages unreproducible because of two reasons. One is that this change was only implemented for the HTML output but not for the LaTeX or Man page output, the other reason is that some upstream prefer to keep timestamps in their documentation so Debian has to carry a patch for these packages forever.

Attached patch fixes both problems, firstly it applies to the HTML, LaTeX, and Man page output at the same time and secondly it allows for packages to still carry timestamps while being reproducible. The patch works by setting a deterministic timestamps through the environment variable SOURCE_DATE_EPOCH which contains the unix epoch as an integer. This environment variable was introduced by the Debian reproducible builds team but it is meant to be used by any distribution [3]. The package help2man is now supporting this [4] we are also persuading txt2man [5], epydoc [6], and GCC [7] to support this.

[1] https://reproducible.debian.net/issues/timestamps_in_documentation_generated_by_doxygen_issue.html
[2] https://reproducible.debian.net/issues/unstable/timestamps_in_manpages_generated_by_doxygen_issue.html
[3] https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal
[4] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=787444
[5] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=790801
[6] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=790899
[7] https://gcc.gnu.org/ml/gcc-patches/2015-06/msg02210.html

On 2015-07-05 13:25:23 +0000, Dimitri van Heesch wrote:

Review of attachment 306856:

Why is the variable read as a 64 bit int, and then checked for being no bigger than 32 bit?
Will this not cause a "year 2038" problem later on?

On 2015-07-05 13:39:55 +0000, marivalenm@gmail.com wrote:

(In reply to Dimitri van Heesch from comment # 1)

Review of attachment 306856 [details] [review]:

Why is the variable read as a 64 bit int, and then checked for being no
bigger than 32 bit?

it is to prepare for when QT does not suffer from the 2038 problem and it also allows for a more precise error reporting because it is used to check if the value of the variable is within the range of what the QT function can parse and to check if the parsing to an unsigned long long was successful.

Will this not cause a "year 2038" problem later on?

Yes Qt currently suffers from the year 2038 problem and there seems to be no function that takes more than an unsigned integer.

On 2015-07-19 10:15:15 +0000, Dimitri van Heesch wrote:

Thanks for the explanation. I've just pushed a variant of your patch to GitHub.
SHA: b31266c

Please check if it works correctly.

On 2015-07-23 14:41:40 +0000, marivalenm@gmail.com wrote:

Hi

(In reply to Dimitri van Heesch from comment # 3)

Thanks for the explanation. I've just pushed a variant of your patch to
GitHub.
https://github.com/doxygen/doxygen/commit/
SHA: b31266c

Please check if it works correctly.

Thank you for the patch, it works fine for a fixed timezone.
Unfortunately since I submitted my first patch the Debian reproducible builds team decided not to export TZ=UTC for all package builds because that breaks some packages.

As a result packages printing timestamps like doxygen must now also take care to not include the user timezone and instead print timestamps in UTC (or any other fixed timezone).

I realized the problem is caused by the call to localtime in the function QDateTime::setTime_t, using gmtime instead of localtime fixes the problem.

Thank you for your effort towards producing reproducible documentation,
akira

On 2015-08-12 07:08:11 +0000, marivalenm@gmail.com wrote:

Created attachment 309101
Produce timezone independent timestamps to have reproducible output

This patch causes doxygen produce timezone independent timestamps (it has already been tested) though it is probably not the best/most correct way to do it.

Please tell me what you think.

Cheers!
akira

On 2015-12-01 12:17:33 +0000, marivalenm@gmail.com wrote:

Hello,

Pinging this request.

Cheers,
akira

On 2015-12-30 10:20:25 +0000, Dimitri van Heesch wrote:

This bug was previously marked ASSIGNED, which means it should be fixed in
doxygen version 1.8.11. Please verify if this is indeed the case. Reopen the
bug if you think it is not fixed and please include any additional information
that you think can be relevant (preferably in the form of a self-contained example).

On 2016-05-09 20:41:59 +0000, Infinity Zero wrote:

Hello, this bug was not fixed correctly. According to the SOURCE_DATE_EPOCH spec [1], "Formatting MUST be deferred until runtime if an end user should observe the value in their own locale or timezone."

However, as Maria pointed out earlier, the current doxygen (including 1.8.11) uses localtime in setTime_t, so the output is in the timezone of the build machine running doxygen, and not the timezone of the reader consuming the final output.

To fix this, one should add a setTimeUTC_t function that is the same as setTime_t, but uses gmtime instead of localtime.

[1] https://reproducible-builds.org/specs/source-date-epoch/

On 2016-05-09 20:51:02 +0000, Infinity Zero wrote:

Created attachment 327542
Use UTC timezone when displaying QDateTimes parsed from SOURCE_DATE_EPOCH

On 2016-05-09 20:51:46 +0000, Infinity Zero wrote:

It looks like I cannot re-open this bug; I suppose you can do that, or shall I create a new one?

On 2016-05-09 21:01:16 +0000, Infinity Zero wrote:

Created attachment 327544
Use UTC timezone when displaying QDateTimes parsed from SOURCE_DATE_EPOCH

I was a bit hasty with that previous patch; of course we don't need to call gmtime twice if that's all we're using.

@doxygen doxygen closed this Jul 2, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.