builds not reproducible

[anarcat]

atheme 6.x could be built reproducibly in debian. starting with 7.x, this is failing because date timestamps and a mysterious build ID have been added.

https://tests.reproducible-builds.org/rb-pkg/experimental/amd64/atheme-services.html

there is still some mercurial code in the main Makefile to generate a serno.h - it's unclear to me if it is being used.

comments filed in charybdis-ircd/charybdis#148 apply here as well, but I am not sure how to patch this, given the __DATE__ macro in use.

[maxteufel]

Reproducible builds are a waste of time - if you already build from source why do you have to check it matches with what upstream (in this case as in the distributor) built?

[anarcat]

@maxteufel i encourage you to review the arguments i brought forward in charybdis-ircd/charybdis#148. you could also review the project on the debian side at: https://wiki.debian.org/ReproducibleBuilds

this is a relatively new approach to distributing security software, pioneered by the Tor project and Debian, but getting more and more traction.

[maxteufel]

reproducible builds are interesting because they allow users to audit the binaries shipped by the
authors to ensure that they match the source code that supposedly implements the software.

Yes. If I already build $software from a copy of the source code then I don't need to verify my build against the one of anyone else.

Instead of wasting time making software distribution supposedly more 'secure', you could rather spend time making a more secure kernel. If you still want to waste your time with reproducible builds - go ahead and submit a pull request.

[kaniini]

Just some background.

@anarcat is the debian maintainer for atheme-services, and I think their issue is worth solving, so reopening.

[maxteufel]

@anarcat: Could you verify f4cd16d works?

The mercurial code seems to be from the time when atheme used it, however it shouldn't work anymore. When building from git, serno.h is currently generated in include/Makefile, using the hash of the last commit.

Edit: 462847b is now f4cd16d

[anarcat]

thanks for the patch! it seems to apply cleanly on 7.2.6, so I'll be testing this shortly.

another issue that came up while trying to improve the debian package was that the aclocal.m4 and autogen.sh files are different in the tarball and the git repository.

The aclocal.m4 file is in the tarball, but not in the git repo:

$ tar jft ../build-area/atheme-services_7.2.6.orig.tar.bz2 atheme-services-7.2.6/aclocal.m4
atheme-services-7.2.6/aclocal.m4
$ git log --oneline --stat --all -- aclocal.m4
b455209 [svn] - revert r8001.
 aclocal.m4 | 22 ----------------------
 1 file changed, 22 deletions(-)
72efb6e [svn] Re-add automake generated files (no, the whole world does not have automake and friends installed, relying on this is not nice)
 aclocal.m4 | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
df9bc89 [svn] Added autogen.sh, removed generated files
 aclocal.m4 | 7131 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------
 1 file changed, 7131 deletions(-)
56a851b [svn] Step 2; reinstate trunk using my tree.
 aclocal.m4 | 7131 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 7131 insertions(+)

The autogen file is in the git repo, but not in the tarball:

$ git log --oneline --stat -1 -- autogen.sh
67e60bf autogen: Remove the cd to $LAST_DIR as well as it's wrong now.
 autogen.sh | 2 --
 1 file changed, 2 deletions(-)
$ LANG=C tar jft atheme-services_7.2.6.orig.tar.bz2 atheme-services-7.2.6/autogen.sh
tar: atheme-services-7.2.6/autogen.sh: Not found in archive
tar: Exiting with failure status due to previous errors
[1028]anarcat@angela:build-area2$ tar jft atheme-services_7.2.6.orig.tar.bz2 | grep autogen
atheme-services-7.2.6/libmowgli-2/autogen.sh

notice how there's a autogen in libmowgli there...

this makes it difficult to have a git repo that follows the upstream releases. we have a weird "overlay" setup where our git repo currently only tracks the "debian/" directory which we overlay on top of atheme's source code. this is an uncommon practice that is used only in such extreme cases because it makes porting patches harder (as the source is not available). i would like to get rid of that clunkiness on our side, but that would mean syncing those files with the tarball.

[maxteufel]

thanks for the patch! it seems to apply cleanly on 7.2.6, so I'll be testing this shortly.

Did you test this now? It would be nice if we could fix this before the next minor release.

For the aclocal/autogen stuff, do the commits in https://github.com/atheme/atheme/tree/fix/makerelease-updates fix your issues?

[anarcat]

unfortunately, the debian package is stuck on 7.0 because of the libmowgli dependency:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818977

i have to clean that up on the debian side before going ahead with the 7.2 packaging...

sorry for the delays, thanks for the autogen reference.

[anarcat]

f4cd16d seems to make builds reproducible, so I have factored it into the debian package. we'll see what the reproducible builds project thinks of the change, hopefully it will agree with me.

regarding autogen, i am not sure about the fix. the problem I am having is that the git tree does not match the tarball. i may be able to workaround the serno.h flag file (by simply injecting the debian package number there at config time), but it is definitely harder for autogen files - so if those are shipped in the git repository or removed from the tarball as well, it would certainly help.

i'll open a separate issue about the build issues when i stumble upon them as well. for now, I am building the debian package as an "overlay" which means I do not work on the complete tarball in our git repository to avoid those issues.