Support defining compilation date in $SOURCE_DATE_EPOCH

[jamessan]

There is an ongoing effort to make FOSS software reproducibly buildable. In order to make Vim build reproducibly, it is necessary to allow defining the date/time that is part of VIM_VERSION_LONG as part of the build process.

This commit enables that by adding support for the SOURCE_DATE_EPOCH spec. When the $SOURCE_DATE_EPOCH environment variable is defined, it will be used to populate the BUILD_DATE preprocessor define.

If BUILD_DATE is not defined, the existing behavior of relying on the preprocessor's __DATE__/__TIME__ symbols will be used.

[brammool]

James McCoy wrote:

There is an ongoing effort0 to make FOSS software reproducibly
buildable. In order to make Vim build reproducibly, it is necessary to
allow defining the date/time that is part of VIM_VERSION_LONG as part of
the build process.

This commit enables that by adding support for the SOURCE_DATE_EPOCH
spec1. When the $SOURCE_DATE_EPOCH environment variable is defined,
it will be used to populate the BUILD_DATE preprocessor define.

If BUILD_DATE is not defined, the existing behavior of relying on the
preprocessor's DATE/TIME symbols will be used.

I do not like picking up environment variables that happen to lay
around. It's too unpredictable. One may set it for one build and then
forget to remove it for another.

Why not add an option, a command line flag to configure?

What is the difference between a professional and an amateur?
The ark was built by an amateur; professionals gave us the Titanic.

/// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\ an exciting new programming language -- http://www.Zimbu.org ///
\ help me help AIDS victims -- http://ICCF-Holland.org ///

[jamessan]

On Thu, Dec 03, 2015 at 12:38:41PM -0800, Bram Moolenaar wrote:

James McCoy wrote:

There is an ongoing effort0 to make FOSS software reproducibly
buildable. In order to make Vim build reproducibly, it is necessary to
allow defining the date/time that is part of VIM_VERSION_LONG as part of
the build process.

This commit enables that by adding support for the SOURCE_DATE_EPOCH
spec1. When the $SOURCE_DATE_EPOCH environment variable is defined,
it will be used to populate the BUILD_DATE preprocessor define.

If BUILD_DATE is not defined, the existing behavior of relying on the
preprocessor's DATE/TIME symbols will be used.

I do not like picking up environment variables that happen to lay
around.

Well, that's the implementation defined in the spec:

The value MUST be exported through the operating system's usual
environment mechanism.

The reason for the spec is to define how one can take advantage of this
for any software project which has implemented it, without having to
know how different projects decided they wanted to handle it.

It's too unpredictable. One may set it for one build and then
forget to remove it for another.

The only thing it affects is the "compiled $date" string in :version.
That seems like an ok risk to take.

Why not add an option, a command line flag to configure?

Because that makes Vim its own unique snowflake, instead of adhering to
the behavior defined by the spec. If someone is trying to rebuild their
entire software stack to verify it matches the binaries they were given,
having to learn N different ways to enable this functionality isn't very
helpful.

Cheers,

James
GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy vega.james@gmail.com

[brammool]

James McCoy wrote:

On Thu, Dec 03, 2015 at 12:38:41PM -0800, Bram Moolenaar wrote:

James McCoy wrote:

There is an ongoing effort0 to make FOSS software reproducibly
buildable. In order to make Vim build reproducibly, it is necessary to
allow defining the date/time that is part of VIM_VERSION_LONG as part of
the build process.

This commit enables that by adding support for the SOURCE_DATE_EPOCH
spec1. When the $SOURCE_DATE_EPOCH environment variable is defined,
it will be used to populate the BUILD_DATE preprocessor define.

If BUILD_DATE is not defined, the existing behavior of relying on the
preprocessor's DATE/TIME symbols will be used.

I do not like picking up environment variables that happen to lay
around.

Well, that's the implementation defined in the spec:

The value MUST be exported through the operating system's usual
environment mechanism.

The reason for the spec is to define how one can take advantage of this
for any software project which has implemented it, without having to
know how different projects decided they wanted to handle it.

It's too unpredictable. One may set it for one build and then
forget to remove it for another.

The only thing it affects is the "compiled $date" string in :version.
That seems like an ok risk to take.

Why not add an option, a command line flag to configure?

Because that makes Vim its own unique snowflake, instead of adhering to
the behavior defined by the spec. If someone is trying to rebuild their
entire software stack to verify it matches the binaries they were given,
having to learn N different ways to enable this functionality isn't very
helpful.

Sorry, I just do not agree with that spec. Building software using
environment variables is brittle and is bound to go wrong. Environment
variables can be set for various reasons, and there is hardly any
control over them. Generally they are user preferences. If you want a
reliable build, you need to use something else, such as command line
arguments.

hundred-and-one symptoms of being an internet addict:
236. You start saving URL's in your digital watch.

/// Bram Moolenaar -- Bram@Moolenaar.net -- http://www.Moolenaar.net \
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\ an exciting new programming language -- http://www.Zimbu.org ///
\ help me help AIDS victims -- http://ICCF-Holland.org ///

[jamessan]

Building software using
environment variables is brittle and is bound to go wrong. Environment
variables can be set for various reasons, and there is hardly any
control over them.

This is a very targeted environment variable, so I don't see why it would be set "for various reasons" or why it's bound to go wrong. The build is already affected by environment variables like CFLAGS, CPPFLAGS, and LDFLAGS, which have far more impact to the build than SOURCE_DATE_EPOCH would.

[mattn]

What is your matter? Different timezone between local and build-server?

[vim-ml]

On Thu, Dec 3, 2015 at 2:22 AM, James McCoy vim-dev-github@256bit.org wrote:

There is an ongoing effort0 to make FOSS software reproducibly
buildable. In order to make Vim build reproducibly, it is necessary to
allow defining the date/time that is part of VIM_VERSION_LONG as part of
the build process.

This commit enables that by adding support for the SOURCE_DATE_EPOCH
spec1. When the $SOURCE_DATE_EPOCH environment variable is defined,
it will be used to populate the BUILD_DATE preprocessor define.

If BUILD_DATE is not defined, the existing behavior of relying on the
preprocessor's DATE/TIME symbols will be used.

---

You can view, comment on, or merge this pull request online at:

#513

Commit Summary

Support defining compilation date in $SOURCE_DATE_EPOCH

File Changes

M src/config.h.in (3)
M src/configure.in (10)
M src/version.c (6)

Patch Links:

https://github.com/vim/vim/pull/513.patch
https://github.com/vim/vim/pull/513.diff

What is the status of that so-called "specification"? AFAICT, this is
the "initial version", IOW, not yet authoritative.

I don't care what some Debian employee says that we MUST, MUST NOT,
SHOULD, etc., do: as far as I'm concerned, the Vim source version is
univocally defined by the major-minor-patchlevel version (currently
7.4.1397) which sends back to a repository tag (v7.4.1397 in both the
git master and the Mercurial mirror). That's as reproducible as anyone
should want. If for any reason I wanted to go back to, oh, anything,
let's say 7.3.25, I could do it very easily by updating my clone to
the corresponding tag and recompiling. The result would behave just
like a Vim 7.3.25 compiled when that patch was published (assuming
that Intel already sold x86_64 processors at the time), including in
the ouput from ":version" in a running Vim, or from "vim --version" on
the command-line.

This is as reproducible as anyone sensible could want; anything more
is intellectual masturbation.

Best regards,
Tony.

[jamessan]

Hi Tony,

I don't care what some Debian employee says that we MUST, MUST NOT,
SHOULD, etc.

Debian doesn't have employees and this is not just a Debian effort, as can be seen at https://reproducible-builds.org/who/. As I stated from the start, this is a
broader effort across the FOSS community.

as far as I'm concerned, the Vim source version is
univocally defined by the major-minor-patchlevel version (currently
7.4.1397) which sends back to a repository tag (v7.4.1397 in both the
git master and the Mercurial mirror). That's as reproducible as anyone
should want.

If all you're concerned about is whether the source is the same as the
canonical source, then yes that's enough. That's not what the reproducible
builds effort is about, though, as can clearly be seen by reading the front
page of the site:

Most aspects of software verification are done on source code, as that is
what humans can reasonably understand. But most of the time, computers
require software to be first built into a long string of numbers to be used.
With reproducible builds, multiple parties can redo this process
independently and ensure they all get exactly the same result. We can thus
gain confidence that a distributed binary code is indeed coming from a given
source code.

This is very important for various reasons but one obvious reason is increased
confidence that people in sensitive situations (such as whistle-blowers or those
under oppressive regimes) aren't getting tampered binaries.

This is as reproducible as anyone sensible could want; anything more
is intellectual masturbation.

If you want to have a constructive discussion, I'll be glad to do so. If not,
please don't waste everyone's time with such responses.

[jamessan]

What is your matter? Different timezone between local and build-server?

No, the issue is that embedding the date/time in the compiled output introduces
unnecessary differences between rebuilds of the source, which makes it
difficult to verify a binary was built from a specific version of the source.

It's understandable that this may not matter to the average Vim user/developer.
However, in situations where that verifiability is important, this patch
enables people to perform reproducible builds while having no impact on people
that don't care about such matters.

[dhduvall]

Well, as the Solaris maintainer of vim, I appreciate this patch. Reproducible binary builds are important to us, and this is a great step. Thank you.

[jamessan]

Rebased to v8.1.0105.

[codecov-io]

Codecov Report

Merging #513 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #513      +/-   ##
==========================================
+ Coverage   82.24%   82.26%   +0.01%     
==========================================
  Files         134      134              
  Lines      147604   147604              
==========================================
+ Hits       121404   121421      +17     
+ Misses      26200    26183      -17

Impacted Files	Coverage Δ
src/version.c	92.1% <ø> (ø)	⬆️
src/if_xcmdsrv.c	85.99% <0%> (-0.18%)	⬇️
src/message.c	79.43% <0%> (-0.05%)	⬇️
src/gui.c	63.78% <0%> (+0.05%)	⬆️
src/os_unix.c	63.55% <0%> (+0.22%)	⬆️
src/ui.c	72.32% <0%> (+1.11%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 559b9c6...b35c809. Read the comment docs.

[brammool]

Looks like this is reasonably widely supported now. Giving a warning somewhere that the date is fixed will help. I would prefer to give a warning when the build date differs from now, but that seems complicated.