-
Notifications
You must be signed in to change notification settings - Fork 7.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reproducible builds difficult due to path names dependencies (IDFGH-3992) #5873
Comments
Thanks for raising this feature request, thanks. |
Hi @m-bruck, I'm glad that you're interested in reproducible builds for ESP-IDF, this is something we're closing in on being able to advertise as supported. At least we're very close to being able to offer this if the build paths match, supporting reproducible builds regardless of build path is an extra challenge - as you've identified.
We can add an option to disable this, as suggested. The purpose of embedding the of the ELF is to provide an easy answer to the question "are we debugging [with gdb/etc] the correct ELF file for the running firmware?", so modifying it in other ways is likely not viable for its intended purpose.
The fix for most of these is to set assertions to "silent" in Kconfig: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/kconfig.html#config-compiler-optimization-assertion-level Otherwise, assertions embed the gcc Unfortunately setting silent assertions is not quite enough to avoid There are a couple of other things to note when aiming for reproducible builds, you can read about these at #5251. Once we actually can support reproducible builds then we'll have a checklist in the documentation, and possible a top-level config item that can be set to ensure all of these are correctly configured. However at the moment this is very much a work-in-progress, and I'm afraid I don't have any ETA that I can give you. |
CMake docs mention a function |
If silent assert configuration is enabled, LWIP asserts are now 'silent' also. Also updates KConfig to note that LWIP asserts are also disabled when asserts are disabled globally (this was already the behaviour, but the config item suggested otherwise.) Progress towards #5873
If silent assert configuration is enabled, LWIP asserts are now 'silent' also. Also updates KConfig to note that LWIP asserts are also disabled when asserts are disabled globally (this was already the behaviour, but the config item suggested otherwise.) Progress towards #5873
Hi @m-bruck, Sorry noone updated you on the progress towards this yet. At this point the items you mentioned should be addressed provided this configuration item is enabled, not necessary to set "silent" asserts or any other changes provided all source paths are under the project directory or the IDF directory. The one outstanding problem is that the ELF file SHA is still embedded into the binary, we still need to add a config option to disable this. |
Just an update on this, the upcoming IDF v5.0 adds a new Kconfig option, CONFIG_APP_REPRODUCIBLE_BUILD. This option also produces same ELF files in every build, by substituting debug paths embedded into the ELF file. We still need to add a section to the build system docs explaining this option, we'll keep this issue open until this is done. |
@igrr Perfect!!! Thanks! |
Docs about reproducible builds where added in 2ddcb13 Marking this issue as resolved. |
Currently reproducible builds require reusing the same build paths for both the IDF as well as the project.
The project path it is emitted in debug sections in the ELF file and thus contaminates the build via the SHA ELF checksum. A quick fix would be a build option to disable the embedding of the checksum by esptool.py. Better yet esptool.py could calculate that particular checksum over its output (obviously before patching it in there) instead of its input.
The IDF-Path is embedded (repeatedly) for assertions in the binary. This might be harder to fix. But as an added benefit this would reduce memory footprint and avoid leaking unrelated information that can be deduced from the absolute path (i.e. user names). As a partial solution it might be possible for IDF_PATH to be relative to the project path, i.e. ..\esp-idf (currently the build fails when attempting this.)
The text was updated successfully, but these errors were encountered: