Reproducible builds difficult due to path names dependencies (IDFGH-3992)

[m-bruck]

Currently reproducible builds require reusing the same build paths for both the IDF as well as the project.

1. The project path it is emitted in debug sections in the ELF file and thus contaminates the build via the SHA ELF checksum. A quick fix would be a build option to disable the embedding of the checksum by esptool.py. Better yet esptool.py could calculate that particular checksum over its output (obviously before patching it in there) instead of its input.

2. The IDF-Path is embedded (repeatedly) for assertions in the binary. This might be harder to fix. But as an added benefit this would reduce memory footprint and avoid leaking unrelated information that can be deduced from the absolute path (i.e. user names). As a partial solution it might be possible for IDF_PATH to be relative to the project path, i.e. ..\esp-idf (currently the build fails when attempting this.)

[Alvin1Zhang]

Thanks for raising this feature request, thanks.

[projectgus]

Hi @m-bruck,

I'm glad that you're interested in reproducible builds for ESP-IDF, this is something we're closing in on being able to advertise as supported. At least we're very close to being able to offer this if the build paths match, supporting reproducible builds regardless of build path is an extra challenge - as you've identified.

The project path it is emitted in debug sections in the ELF file and thus contaminates the build via the SHA ELF checksum. A quick fix would be a build option to disable the embedding of the checksum by esptool.py. Better yet esptool.py could calculate that particular checksum over its output (obviously before patching it in there) instead of its input.

We can add an option to disable this, as suggested.

The purpose of embedding the of the ELF is to provide an easy answer to the question "are we debugging [with gdb/etc] the correct ELF file for the running firmware?", so modifying it in other ways is likely not viable for its intended purpose.

The IDF-Path is embedded (repeatedly) for assertions in the binary.

The fix for most of these is to set assertions to "silent" in Kconfig: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/kconfig.html#config-compiler-optimization-assertion-level

Otherwise, assertions embed the gcc __FILE__ macro and as far as I know CMake will always generate an absolute path on the gcc command line regardless of what is used in CMake.

Unfortunately setting silent assertions is not quite enough to avoid __FILE__ being built into the final binary at all, it looks like some parts of FreeRTOS and LWIP still use it. Will take a look at what we can do to support __FILE__ not being included at all.

There are a couple of other things to note when aiming for reproducible builds, you can read about these at #5251. Once we actually can support reproducible builds then we'll have a checklist in the documentation, and possible a top-level config item that can be set to ensure all of these are correctly configured.

However at the moment this is very much a work-in-progress, and I'm afraid I don't have any ETA that I can give you.

[m-bruck]

CMake docs mention a function file(RELATIVE_PATH <variable> <directory> <file>), perhaps that could be used to strip IDF_PATH and pass a relative path as a preprocessor define (e.g. __REL_FILE__) that can be used by assertions.

[projectgus]

Hi @m-bruck,

Sorry noone updated you on the progress towards this yet.

At this point the items you mentioned should be addressed provided this configuration item is enabled, not necessary to set "silent" asserts or any other changes provided all source paths are under the project directory or the IDF directory.

The one outstanding problem is that the ELF file SHA is still embedded into the binary, we still need to add a config option to disable this.

[igrr]

Just an update on this, the upcoming IDF v5.0 adds a new Kconfig option, CONFIG_APP_REPRODUCIBLE_BUILD. This option also produces same ELF files in every build, by substituting debug paths embedded into the ELF file.

We still need to add a section to the build system docs explaining this option, we'll keep this issue open until this is done.

[alezancomelit]

@igrr Perfect!!! Thanks!

[ESP-Marius]

Docs about reproducible builds where added in 2ddcb13

Marking this issue as resolved.