Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong src RPM deps parsing by rpmbuild #2690

Closed
socketpair opened this issue Oct 3, 2023 · 15 comments
Closed

Wrong src RPM deps parsing by rpmbuild #2690

socketpair opened this issue Oct 3, 2023 · 15 comments

Comments

@socketpair
Copy link
Contributor

socketpair commented Oct 3, 2023

rpm -qRp /home/fedora/rpmbuild/SRPMS/frr-8.5.2-2.1696341206.16.8ff8ce50.utm.src.rpm:

gives the following:

...
perl-XML-LibXML
perl-generators
python3-devel
...

That's OK. Nothing about pkgconfig.

Now, try to build it:

rpmbuild --verbose --debug -rr /home/fedora/rpmbuild/SRPMS/frr-8.5.2-2.1696341206.16.8ff8ce50.utm.src.rpm

Last login: Tue Oct  3 14:43:10 UTC 2023 on pts/0
Installing /home/fedora/rpmbuild/SRPMS/frr-8.5.2-2.1696341206.16.8ff8ce50.utm.src.rpm
D: loading keyring from rpmdb
D: PRAGMA secure_delete = OFF: 0
D: PRAGMA case_sensitive_like = ON: 0
D:  read h#       1 
Header SHA256 digest: OK
Header SHA1 digest: OK
D: added key gpg-pubkey-5323552a-6112bcdc to keyring
D: /home/fedora/rpmbuild/SRPMS/frr-8.5.2-2.1696341206.16.8ff8ce50.utm.src.rpm: Header SHA256 digest: OK
D: /home/fedora/rpmbuild/SRPMS/frr-8.5.2-2.1696341206.16.8ff8ce50.utm.src.rpm: Header SHA1 digest: OK
D:   install: frr-8.5.2-2.1696341206.16.8ff8ce50.utm.x86_64 has 13 files
D: create     100644  1 (1000,1000)  1364 0000-remove-babeld-and-ldpd.patch;651c292f
D: create     100644  1 (1000,1000)  1740 0002-enable-openssl.patch;651c292f
D: create     100644  1 (1000,1000)  6746 0003-disable-eigrp-crypto.patch;651c292f
D: create     100644  1 (1000,1000)  3619 0004-fips-mode.patch;651c292f
D: create     100644  1 (1000,1000)   870 0005-remove-grpc-test.patch;651c292f
D: create     100644  1 (1000,1000)  1619 0007-ideco-route-table.patch;651c292f
D: create     100644  1 (1000,1000)9881857 frr-8.5.2.tar.gz;651c292f
D: create     100644  1 (1000,1000)   185 frr-sysusers.conf;651c292f
D: create     100644  1 (1000,1000)    26 frr-tmpfiles.conf;651c292f
D: create     100644  1 (1000,1000)  1633 frr.fc;651c292f
D: create     100644  1 (1000,1000)  4177 frr.if;651c292f
D: create     100644  1 (1000,1000) 16037 frr.spec;651c292f
D: create     100644  1 (1000,1000)  3511 frr.te;651c292f
setting SOURCE_DATE_EPOCH=1696204800
D: ========== +++ frr-8.5.2-2.1696341206.16.8ff8ce50.utm x86_64/linux 0x0
.......
D:  read h#     474 
Header V4 RSA/SHA256 Signature, key ID 5323552a: OK
Header SHA256 digest: OK
Header SHA1 digest: OK
D:  Requires: perl-generators                               YES (db provides)
D:  Requires: pkgconfig(systemd)                            NO    <<<<======================================
D:  read h#     480 
Header V4 RSA/SHA256 Signature, key ID 5323552a: OK
Header SHA256 digest: OK
Header SHA1 digest: OK
D:  Requires: python3-devel                                 YES (db provides)
...
error: Failed build dependencies:
	pkgconfig(systemd) is needed by frr-8.5.2-2.1696341206.16.8ff8ce50.utm.x86_64

So, WHY? From one point of view it's not in dependencies, from another – it is.

RPM version 4.18.1

File is attached. Sorry for zip. GitHub does not allow attaching just rpm.

frr-8.5.2-2.1696341206.16.8ff8ce50.utm.src.zip

@socketpair
Copy link
Contributor Author

Moreover, I don't understand how it appears in this srpm file. This file was made in our build scripts.

@pmatilai
Copy link
Member

pmatilai commented Oct 4, 2023

The dependencies recorded in an src.rpm are only valid for the environment (including any cli-switches) it was generated in. It's entirely possible to end up with something quite different on a spec reparse, which is what happens on any build.

As for pkg-config deps, I see this in the spec, which makes me wonder:

# .........
# BuildRequires:  pkgconfig(systemd)
# BuildRequires: selinux-policy-devel

I can't reproduce any pkgconfig deps appearing in the build, and I can't imagine rpm hallucinating them up either (this is certainly not AI 😅 ). I suggest you double check your findings.

@pmatilai
Copy link
Member

pmatilai commented Oct 4, 2023

BTW, it's also technically possible to (externally) rewrite the specfile after launching a build on it, and end up with wildly different in the src.rpm than what the build was launched with. It would be a nasty thing to do of course, but technically possible, and could explain oddities like this when external build scripts are involved.

@socketpair
Copy link
Contributor Author

socketpair commented Oct 4, 2023

@pmatilai Commented things – are my lines – I did not remove after experiments. Yes, our scripts build .src.rpm from .spec file and then (in the same environment) install build deps. The problem that rpmbuild -rr says about missing deps, but dnf builddep does not install them (does not see them).

Yes, I have double-checked. And it is easily reproducible. We definitely DO NOT rewrite spec-file. We don't change .src.rpm file after it has been built.

Tell me how to debug the issue. frr is the only one package that has such problem.

Yes, I understand it looks like some stupid and impossible thing, but I can not find the source of the problem.

frr-8.5.2-2.1696412835.16.8ff8ce50.utm.src.zip

Removed comments and built .src.rpm again. Everything is exactly the same.

We build .src.rpm by this command: rpmbuild -bs /home/fedora/rpmbuild/SPECS/frr.spec --rmspec nothing special.

Anyway. rpm -qRp shows one result, and rpmbuild --verbose --debug -rr different. I mean regarding dependencies. On the SAME file. Isn't it a bug ? None of them parse embedded .spec, right ?

@pmatilai
Copy link
Member

pmatilai commented Oct 4, 2023

The one thing -bs and -br will disagree with is dynamic buildrequires from %generate_buildrequires. I don't see that in the src.rpm but I don't know what macros and stuff you may have locally.

@pmatilai
Copy link
Member

pmatilai commented Oct 4, 2023

I suggest you try reproducing it in a different environment. A clean container image or such.

@pmatilai
Copy link
Member

pmatilai commented Oct 4, 2023

Oh, I see it now. Try 'rpmspec --parse frr.spec' to see how it looks to rpm, and it becomes quite clear.

The pkgconf(systemd) dependency comes from the %{?selinux_requires} macro. And if rpmbuild -bs is executed in an environment where that macro is not defined then it will miss those defines and no error message will be emitted because it's a conditional macro like that.

No bug here.

@pmatilai pmatilai closed this as completed Oct 4, 2023
@socketpair
Copy link
Contributor Author

socketpair commented Oct 4, 2023

@pmatilai sorry for bash script, but I want just show what happens in our build script:

unpriv() {
    su \
        --login \
        --whitelist-environment=OUR_DEBUG \
        --shell=/usr/bin/env \
        "$BUILD_USER" \
        -- \
        "PATH=$PATH" \
        "HOME=/home/$BUILD_USER" \
        "$@"
}

.....

    unpriv rpmbuild "${defines[@]}" -bs "$spec_file" --rmspec
    srpm=("$RPMBUILD"/SRPMS/*.src.rpm)
    while :; do
        unpriv rpmbuild "${defines[@]}" -rr "$srpm" && exit_code=0 || exit_code=$?
        case "$exit_code" in
            0)
                break
                ;;
            11)
                reqsrpm=("$RPMBUILD"/SRPMS/*.buildreqs.nosrc.rpm)
                if [ -f "$reqsrpm" ]; then
                    dnf -y builddep --srpm "$reqsrpm"
                    rm -f "$reqsrpm"
                else
                    dnf -y builddep --srpm "$srpm"  # <========= FALLS HERE because -rr DOES NOT generate.buildreqs.nosrc.rpm  with "pkgconfig(systemd)"
                fi
                ;;
            *)
                echo "rpmbuild -rr FAILED code $exit_code" >&2
                exit 42
                ;;
        esac
    done
    

unpriv is just a wrapper that drops permissions, as this part of script run under root.

And on this package it falls into an endless loop. Note, environment is exactly the same on both rpmbuild runs.

@socketpair
Copy link
Contributor Author

socketpair commented Oct 4, 2023

@pmatilai sorry for not explaining clearly. Seems there are two bugs.

  1. When -rr says error about missing deps (and exits with code 11) but does not generate .buildreqs.nosrc.rpm in some conditions. In generally – yes, this functionality works as expected, but not this time... Possibly because there are no dynamic deps macro? If yes, it should be mentioned in documentation, and it should not exit with code 11, as I think.

  2. -rr sees not installed deps, but dnf builddep does not see. It happens on the first iteration – dnf installs all deps except this pkgconfig(systemd).

@socketpair
Copy link
Contributor Author

socketpair commented Oct 4, 2023

Interesting... -rr --nodeps rewrites .src.rpm in a way that dnf INSTALLS pkgconfig(systemd) afterwards.

@pmatilai
Copy link
Member

pmatilai commented Oct 4, 2023

Rpm doesn't use the dependencies recorded into an src.rpm in any circumstance, the spec is always reparsed. I don't know what the difference between rpmbuild -bs and -br is in your environment, but that's where the issue is.

@socketpair
Copy link
Contributor Author

socketpair commented Oct 4, 2023

@pmatilai there is no difference. Really. I have provided part of the shell script. Tell me how to debug, please. I strongly consider there is a bug in RPM, but can't knock it down.

@socketpair
Copy link
Contributor Author

Seems I figured out. After installing deps from file generated by -bs some macroses appear. that's what is changed in environment.

So the bug is: not generating .buildreqs.nosrc.rpm. If it was generated, bug would not happen.

@socketpair
Copy link
Contributor Author

Seems dup of #797

@socketpair
Copy link
Contributor Author

#781

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants