Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reproducible builds #510

Open
guseggert opened this issue Dec 6, 2021 · 3 comments
Open

Reproducible builds #510

guseggert opened this issue Dec 6, 2021 · 3 comments
Labels
dif/expert Extensive knowledge (implications, ramifications) required effort/weeks Estimated to take multiple weeks kind/enhancement A net-new feature or improvement to an existing feature need/analysis Needs further analysis before proceeding P2 Medium: Good to have, but can wait until someone steps up

Comments

@guseggert
Copy link
Contributor

Currently, two builds on identical source trees result in different CIDs because of at least two issues (possibly more):

  • The dist.json and build-info files contain dates + timestamps
  • We don't strip timestamps from file metadata when archiving them, and the gzip header also contains a timestamp, causing their bytes to be different even though the contents are the same

From spot-checking a build, the binaries themselves are identical once they are extracted from the archive.
@guseggert guseggert added need/triage Needs initial labeling and prioritization dif/easy Someone with a little familiarity can pick up effort/days Estimated to take multiple days, but less than a week P2 Medium: Good to have, but can wait until someone steps up and removed need/triage Needs initial labeling and prioritization labels Dec 6, 2021
@guseggert
Copy link
Contributor Author

guseggert commented Dec 6, 2021 •
edited
Loading

Two examples that contain identical source code:

  • /ipfs/bafybeihlvvrmxwpdwsksy7lqdm63jfenuekfzxyn74oymd2xpg5w6k5pjy/go-ipfs/v0.11.0-rc2
  • /ipfs/bafybeie5tqusm4kdnssq2mzkzgvh3ltt46lupuqkplbft6ylnysq4czkde/go-ipfs/v0.11.0-rc2

@guseggert
Copy link
Contributor Author

see also ipfs/kubo#7848

@ipfs ipfs deleted a comment from welcome bot Nov 28, 2022
@lidel
Copy link
Member

lidel commented Nov 28, 2022

Related: https://warpforge.io was announced during IPFS Camp this year:

We’re offering software build tools for language-agnostic, hermetic builds, for developer productivity, security, and sanity. It’s tooling for developers. It’s open source. It gives reproducible environments, so you can do reproducible work

Warpforge — Hashes go in, hashes come out, exec in the middle! - Eric Evenchick:

Warpforge is a tool for building software and creating data pipelines, founded in content addressable primitives and aimed at happily operating in a decentralized environment — both in the sense of “on laptops as well as in datacenters”, as well as in the sense of “I share build instructions with friends, and we don’t need a monorepo to coordinate”. Along the way, we put may IPLD datastructures to use to obtain our goals — including some data structures which are used to create local solutions to the infamous decentralized naming problem. This talk was given at IPFS Camp 2022 in Lisbon, Portugal.

Feels like something worth evaluating for "dist.ipfs.tech 2.0"

@lidel lidel added kind/enhancement A net-new feature or improvement to an existing feature need/analysis Needs further analysis before proceeding dif/expert Extensive knowledge (implications, ramifications) required effort/weeks Estimated to take multiple weeks and removed dif/easy Someone with a little familiarity can pick up effort/days Estimated to take multiple days, but less than a week labels Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
dif/expert Extensive knowledge (implications, ramifications) required effort/weeks Estimated to take multiple weeks kind/enhancement A net-new feature or improvement to an existing feature need/analysis Needs further analysis before proceeding P2 Medium: Good to have, but can wait until someone steps up
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lidel @guseggert