Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFE: save the parsed spec in src.rpm #1241

Closed
pmatilai opened this issue May 26, 2020 · 7 comments · Fixed by #2047
Closed

RFE: save the parsed spec in src.rpm #1241

pmatilai opened this issue May 26, 2020 · 7 comments · Fixed by #2047
Labels

Comments

@pmatilai
Copy link
Member

With the trend of increasingly dynamic content generation for specs (currently via complex macros, in future automatic & dynamic sub packages, #329, #1222 etc) the originating spec and the parsed one might differ by hundreds of lines, making it really hard to grasp how we got from A to B. It seems we should store the parsed spec in the src.rpm somehow. IIRC this has actually been requested before but can't find the reference atm.

@ffesti
Copy link
Contributor

ffesti commented May 26, 2020

rpmSpecGetSection() returns it when passed RPMBUILD_NONE. Guess adding this to the API is what you remember even if it has been a while:
c38504c4ba build/rpmspec.h (Panu Matilainen 2011-06-17 13:44:39 +0300 78) * As a special case, RPMBUILD_NONE as section returns the entire spec in c38504c4ba build/rpmspec.h (Panu Matilainen 2011-06-17 13:44:39 +0300 79) * preprocessed (macros expanded etc) format.

@pmatilai
Copy link
Member Author

No, that was simply to support rpmspec --parse. There was a request to add the spec to the header, or something, someplace, but whether that request was for the parsed spec I don't recall.

@Conan-Kudo
Copy link
Member

There was a request years ago (not sure it was tracked in rpm.org trac) for doing this for reproducible builds, because we want to be able to replay the exact build environment and settings, and the current way is incomplete.

@nim-nim
Copy link

nim-nim commented Jul 6, 2020

Generally speaking it would be great to store more build-time information in the spec. I’ve been requested to store the buildroot state for debuging and reproducibility needs, for example.

https://src.fedoraproject.org/fork/nim/rpms/redhat-rpm-config/c/bc4e6a79355f3a2b73abeea7e5c0e1f53b09579e?branch=forge-with-patches-auto-call-auto-changelog-auto-srpm-bump

shows how to store more build-time info in the SRPMs using current unchanged rpm. So it does not even need rpm engine changes.

However one could argue that writing 13 bytes of spaces in sources files in %sourcelist, just so they exist when %build needs to write to them, is not the most elegant solution.

Also calling rpm or rpmspec from within an rpmbuild session is probably not the best of things.

@pmatilai
Copy link
Member Author

As a note to self, we could/should probably use RPMFILE_ARTIFACT to flag such files.

@pmatilai pmatilai added the RFE label Apr 14, 2022
pmatilai added a commit to pmatilai/rpm that referenced this issue May 3, 2022
…nt#1241)

The trend of packaging is to hide more and more of what actually happens
behind helper macros and other dynamically generated constructs. While
this is mostly a good thing, it makes understanding and postmortem
analysis harder than it should be. Add the spec in its parsed and
expanded form into the src.rpm header to make the actual contents of the
build more trackable.

It can be argued this does not belong in the header and should be in
payload instead, my rationale for the placement is that this way the
payload remains effectively arch independent, whereas the header already
heavily reflects the particular environment where it was built.

Fixes: rpm-software-management#1241
pmatilai added a commit to pmatilai/rpm that referenced this issue May 3, 2022
…nt#1241)

The trend of packaging is to hide more and more of what actually happens
behind helper macros and other dynamically generated constructs. While
this is mostly a good thing, it makes understanding and postmortem
analysis harder than it should be. Add the spec in its parsed and
expanded form into the src.rpm header to make the actual contents of the
build more trackable.

It can be argued this does not belong in the header and should be in
payload instead, my rationale for the placement is that this way the
payload remains effectively arch independent, whereas the header already
heavily reflects the particular environment where it was built.

Probably worth noting that %setup and %patch built-in macros are not
properly expanded in the stored spec, but that's a separate issue
related the special way %prep is processed.

Fixes: rpm-software-management#1241
pmatilai added a commit to pmatilai/rpm that referenced this issue May 5, 2022
…nt#1241)

The trend of packaging is to hide more and more of what actually happens
behind helper macros and other dynamically generated constructs. While
this is mostly a good thing, it makes understanding and postmortem
analysis harder than it should be. Add the spec in its parsed and
expanded form into the src.rpm header to make the actual contents of the
build more trackable.

It can be argued this does not belong in the header and should be in
payload instead, my rationale for the placement is that this way the
payload remains effectively arch independent, whereas the header already
heavily reflects the particular environment where it was built.

Probably worth noting that %setup and %patch built-in macros are not
properly expanded in the stored spec, but that's a separate issue
related the special way %prep is processed.

Fixes: rpm-software-management#1241
ffesti pushed a commit that referenced this issue May 9, 2022
The trend of packaging is to hide more and more of what actually happens
behind helper macros and other dynamically generated constructs. While
this is mostly a good thing, it makes understanding and postmortem
analysis harder than it should be. Add the spec in its parsed and
expanded form into the src.rpm header to make the actual contents of the
build more trackable.

It can be argued this does not belong in the header and should be in
payload instead, my rationale for the placement is that this way the
payload remains effectively arch independent, whereas the header already
heavily reflects the particular environment where it was built.

Probably worth noting that %setup and %patch built-in macros are not
properly expanded in the stored spec, but that's a separate issue
related the special way %prep is processed.

Fixes: #1241
pmatilai added a commit to pmatilai/rpm that referenced this issue Jun 28, 2022
…nt#1241)

The trend of packaging is to hide more and more of what actually happens
behind helper macros and other dynamically generated constructs. While
this is mostly a good thing, it makes understanding and postmortem
analysis harder than it should be. Add the spec in its parsed and
expanded form into the src.rpm header to make the actual contents of the
build more trackable.

It can be argued this does not belong in the header and should be in
payload instead, my rationale for the placement is that this way the
payload remains effectively arch independent, whereas the header already
heavily reflects the particular environment where it was built.

Probably worth noting that %setup and %patch built-in macros are not
properly expanded in the stored spec, but that's a separate issue
related the special way %prep is processed.

Fixes: rpm-software-management#1241
pmatilai added a commit that referenced this issue Jun 28, 2022
The trend of packaging is to hide more and more of what actually happens
behind helper macros and other dynamically generated constructs. While
this is mostly a good thing, it makes understanding and postmortem
analysis harder than it should be. Add the spec in its parsed and
expanded form into the src.rpm header to make the actual contents of the
build more trackable.

It can be argued this does not belong in the header and should be in
payload instead, my rationale for the placement is that this way the
payload remains effectively arch independent, whereas the header already
heavily reflects the particular environment where it was built.

Probably worth noting that %setup and %patch built-in macros are not
properly expanded in the stored spec, but that's a separate issue
related the special way %prep is processed.

Fixes: #1241
@bmwiedemann
Copy link
Contributor

bmwiedemann commented Jan 6, 2023

I found that unfortunately this change in rpm-4.18 makes many of our rpm builds unreproducible, because our .spec files contain things like make %{?_smp_mflags} and that gets expanded depending on the build machine CPU core count.

Is there an easy way to disable this or do you have a different proposal of how to build bit-identical rpms again?

@pmatilai
Copy link
Member Author

pmatilai commented Jan 9, 2023

Please file a new issue on that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants