Reproducible builds (e.g. via Nix / Bazel)

[tomtau]

Currently, building of unsigned binaries may not be entirely deterministic. It'd be good to port the build infrastructure to use tools that enable reproducible builds, such as Nix or Bazel.
It'd require to port the build infrastructure of https://github.com/intel/linux-sgx as well.

[tomtau]

Here's a bounty on this issue if anyone is interested in attempting it: https://www.bountysource.com/issues/68776656-reproducible-builds-e-g-via-nix-bazel

[bert-tenjy]

Here is a suggestion that if valid would require only a minimal change in the existing build infrastructure to guarantee deterministic generation of binaries.

GCC has an option -frandom-seed=string which, to quote the GCC manual, "can be used to produce reproducibly identical object files".

So, changing line 99 in buildenv.mk of the linux-sgx repository to:
COMMON_FLAGS += -ffunction-sections -fdata-sections -frandom-seed=$@
should result in deterministic generation of all binaries in the project.

Does this solve the issue?

[elichai]

Why do you need to port linux-sgx tools for this?
I think if it's possible to make a deterministic build in rust(which i'm not sure is: rust-lang/rust#34902) it should be possible to combine that with something like gitian to make a full deterministic build

[bert-tenjy]

Why do you need to port linux-sgx tools for this?
I think if it's possible to make a deterministic build in rust(which i'm not sure is: rust-lang/rust#34902) it should be possible to combine that with something like gitian to make a full deterministic build

Clarifying my previous post:

My response is narrowly focused on GCC whose manual explains how to obtain deterministic outputs from compilation. The point I should have made is that in order to reproduce any binary using GCC, each object file on which the binary depends and the binary itself must be compiled with the flag -frandom-seed=string set exactly the same way in each compilation. I believe that -frandom-seed=$@ as one of the options to GCC does just that.

[tomtau]

@elichai so that one can deterministically reproduce the full build environment from scratch, e.g. Edger8r tool and all external C libs.
The current build environment through the Docker image may not be deterministic: https://github.com/baidu/rust-sgx-sdk/blob/master/dockerfile/Dockerfile.1804.stable#L36

[tomtau]

Another related issue: rust-lang/rust#34902

[tomtau]

Google's Asylo has some SGX SDK-related Bazel rules:
https://github.com/google/asylo/blob/master/asylo/bazel/sgx_deps.bzl
https://github.com/google/asylo/tree/master/asylo/distrib/sgx_x86_64

[tomtau]

@dingelish is anyone working on this?
One other side-benefit of this will be to have a bit more modular build procedure / setting -- currently, all the rust-sgx-sdk projects copy-paste and tweak Makefiles (with dozens of environment variables) and build.rs

[tomtau]

Latest SDK 2.6 (#141) is using Nix in its Docker build image: intel/linux-sgx@c505e61#diff-8e95e7bcc798d658d53ee4e52104a762

[ghost]

Hey I have experience in Rust projects with Bazel. Started to port this repo slowly. Would bazel builds still be interesting to you guys?

That being said, it would be one of my first open source contributions I do, so some minimal guidance might be needed.

Would you like me to create merge request as I go, or at the end when the full repo has been moved to bazel?

[tomtau]

I think Bazel builds could improve the current workflows.
Perhaps PRs "as you go" may be better (to get some early feedback and sync up on other changes)?

[ghost]

Alright happy to hear this. Question tho... Default for bazel is to unify the Cargo dependencies into a single file inside third_party folder. That means all crates will source dependencies from the same list. Would this be fine for this project?

I will vendor all dependencies so builds should be fully deterministic. To add new dependency we will depend on cargo-raze tho.

Will try to push a first PR today

[tomtau]

In theory, it should be fine or perhaps even desired.
In practice, there could be some hiccups -- for example, some of the crates patch dependencies: 7e2705f

@dingelish any thoughts?

[dingelish]

My idea is to provide another sample code like "mutual-ra-bazel" "wasmi-bazel" which utilizes bazel for deterministic build for the first stage. Don't know if we should switch all samples to bazel.

[ghost]

Should we live this issue open? Till the port is fully finished?

[tomtau]

Yes

[ghost]

Hey guys, sorry I haven't been pushing anything new. Super busy at work for another two days. After that I plan to finish this. Haven't forgotten about this issue.

[tomtau]

No worries

[sbellem]

I am curious to know what's the status on this, as I am interested in helping.

[tomtau]

@sbellem some initial work using Bazel by @Sythanis:
#156
#151

afaik some of the planned things, such as having "deterministically built" samples ("mutual-ra-bazel", "wasmi-bazel", ...) with Bazel, haven't materialised yet, so feel free to take it from there

[sbellem]

Ok thanks @tomtau!

[sbellem]

@tomtau @dingelish

fyi, I'm still planning to help with this issue. The current status is that I've prepared a draft PR (NixOS/nixpkgs#126990) on the NixOS/nixpkgs repository to add an sgxsdk nix package. This is a prerequisite to build reproducible enclave builds with the rust sgx sdk. This PR is probably going to take a little while to get through, but I should nevertheless be able to make some progress with the rust sgx sdk in the meantime.

[combatpoodle]

@tomtau I’m a build/software/hardware engineer just looking for a project… This looks decent. Is the bounty still applicable? Thanks!

[sbellem]

fyi, been very slow, but working on this still -- a small example is at https://github.com/initc3/nix-sgx-hello-rust

Main issue right now is to build with xargo, but building without xargo works. The proof-of-concept can be run with docker compose -- see readme.