Add disfavored packages to the goal (RhBug:1699672) #1774

j-mracek · 2021-06-02T18:34:56Z

It disfavor packages for solver and remove them from the list of
candidates for weak deps. It will prevents of installing disfavored
packages that are recommended or supplemented.

Requires: rpm-software-management/libdnf#1257
Requires: rpm-software-management/ci-dnf-stack#1005

Alternative to: #1771

j-mracek · 2021-06-04T06:19:18Z

Examples how to use it:
I would like to install pkgA that recommends pkgB without pkgB and ensure that after next reinstall, upgrade, downgrade of pkgA, pkgB will be not installed as weakdep.

Set the rule only for the transaction

dnf install pkgA --setopt=disfavor=pkgB

When config option is disfavor_unmet_weak_deps is True (default) then reinstall, upgrade and so on of pkgA will not bring pkgB as weak dependency.

When config option is disfavor_unmet_weak_deps is False, then pkgB will be installed.
dnf reinstall pkgA --setopt=disfavor_unmet_weak_deps=False

When config option is disfavor_unmet_weak_deps is False, and pkgB disfavored then pkgB will be notinstalled.
dnf reinstall pkgA --setopt=disfavor_unmet_weak_deps=False --setopt=disfavor=pkgB

Set the rule permanently in `/etd/dnf/dnf.conf`

Open /etd/dnf/dnf.conf and in [main] section add or edit key disfavor=pkgB. Values must be comma separated (weakexcludes=pkgB,pkgC)

Set the rule permanently in `/etd/dnf/dnf.conf` using `config-manager`

dnf config-manager --installroot /tmp/testweak --setopt=disfavor=pkgB --save
Note - config-manager always replace original values in dnf.conf

adelton · 2021-06-04T10:37:35Z

@j-mracek, thanks for the code.

To make sure I understand the intended usage correctly -- if I don't have any disfavor set, should the default

dnf upgrade --setopt=disfavor_unmet_weak_deps=True

and

dnf upgrade --setopt=disfavor_unmet_weak_deps=False

behave exactly the same, or differently? On one of my systems, I see

# dnf upgrade --setopt=disfavor_unmet_weak_deps=True
[...]
Transaction Summary
================================================================================
Install    4 Packages
Upgrade  119 Packages
Remove     3 Packages

Total size: 258 M
Total download size: 186 M
Is this ok [y/N]: no
Operation aborted.

and

# dnf upgrade --setopt=disfavor_unmet_weak_deps=False
[...]
Transaction Summary
================================================================================
Install    8 Packages
Upgrade  123 Packages
Remove     3 Packages

Total size: 260 M
Total download size: 187 M
Is this ok [y/N]: no
Operation aborted.

and I wonder if that is expected.

adelton · 2021-06-04T10:46:09Z

I see a weird discrepancy:

# rpm -q gdb exiv2-libs
gdb-10.1-14.fc34.x86_64
exiv2-libs-0.27.3-6.fc34.x86_64
# rpm -q --recommends exiv2-libs
exiv2 = 0.27.3-6.fc34
# rpm -q --recommends gdb
dnf-command(debuginfo-install)
gcc-gdb-plugin(x86-64)
# rpm -q exiv2 gcc-gdb-plugin
package exiv2 is not installed
package gcc-gdb-plugin is not installed

# dnf upgrade gdb exiv2-libs --setopt=disfavor_unmet_weak_deps=True
Last metadata expiration check: 0:14:23 ago on Fri 04 Jun 2021 12:30:05 PM CEST.
Dependencies resolved.
================================================================================
 Package             Architecture  Version                 Repository      Size
================================================================================
Upgrading:
 exiv2-libs          x86_64        0.27.3-7.fc34           updates        765 k
 gdb                 x86_64        10.2-1.fc34             updates        127 k
 gdb-headless        x86_64        10.2-1.fc34             updates        4.0 M
Installing weak dependencies:
 exiv2               x86_64        0.27.3-7.fc34           updates        973 k

Transaction Summary
================================================================================
Install  1 Package
Upgrade  3 Packages

Total size: 5.9 M
Is this ok [y/N]: no
Operation aborted.
# dnf upgrade gdb exiv2-libs --setopt=disfavor_unmet_weak_deps=False
Last metadata expiration check: 0:14:34 ago on Fri 04 Jun 2021 12:30:05 PM CEST.
Dependencies resolved.
================================================================================
 Package               Architecture  Version               Repository      Size
================================================================================
Upgrading:
 exiv2-libs            x86_64        0.27.3-7.fc34         updates        765 k
 gdb                   x86_64        10.2-1.fc34           updates        127 k
 gdb-headless          x86_64        10.2-1.fc34           updates        4.0 M
Installing weak dependencies:
 exiv2                 x86_64        0.27.3-7.fc34         updates        973 k
 gcc-gdb-plugin        x86_64        11.1.1-1.fc34         updates        140 k

Transaction Summary
================================================================================
Install  2 Packages
Upgrade  3 Packages

Total size: 6.0 M
Is this ok [y/N]: no
Operation aborted.

I'm not sure I understand why upgrade of exiv2-libs pulls in exiv2 no matter what, while upgrade of gdb responds to the disfavor_unmet_weak_deps value.

In either of those cases, no disfavor value is set.

This was with

dnf-4.7.1-1.git.9417.0c60438.fc34.noarch
libdnf-0.64.0-1.git.3105.c1dbc8b.fc34.x86_64

j-mracek · 2021-06-04T10:47:28Z

@j-mracek, thanks for the code.

To make sure I understand the intended usage correctly -- if I don't have any disfavor set, should the default

dnf upgrade --setopt=disfavor_unmet_weak_deps=True

and

dnf upgrade --setopt=disfavor_unmet_weak_deps=False

behave exactly the same, or differently? On one of my systems, I see

# dnf upgrade --setopt=disfavor_unmet_weak_deps=True
[...]
Transaction Summary
================================================================================
Install    4 Packages
Upgrade  119 Packages
Remove     3 Packages

Total size: 258 M
Total download size: 186 M
Is this ok [y/N]: no
Operation aborted.

and

# dnf upgrade --setopt=disfavor_unmet_weak_deps=False
[...]
Transaction Summary
================================================================================
Install    8 Packages
Upgrade  123 Packages
Remove     3 Packages

Total size: 260 M
Total download size: 187 M
Is this ok [y/N]: no
Operation aborted.

and I wonder if that is expected.

Yes this is expected. disfavor_unmet_weak_deps=True automatically detects unmet weak deps of installed packages. Even with disfavor_unmet_weak_deps=True anything described in configuration as disfavor it is also disfavored. It means that the patch provides two parallel mechanisms for disfavor packages.

adelton · 2021-06-04T10:49:07Z

Awesome, thanks. In that case, could you please look into the exiv2-libs / exiv2 case? That seems to avoid the disfavor_unmet_weak_deps=True logic.

adelton · 2021-06-04T11:13:40Z

Another use-case where disfavor_unmet_weak_deps=True does not seem to kick in is a supplements on uninstalled package in repository:

# rpm -q gawk gawk-all-langpacks
gawk-5.1.0-3.fc34.x86_64
package gawk-all-langpacks is not installed
# dnf repoquery --supplements gawk-all-langpacks
Last metadata expiration check: 0:41:10 ago on Fri 04 Jun 2021 12:30:05 PM CEST.
gawk = 5.1.0-3.fc34
# dnf reinstall gawk --setopt=disfavor_unmet_weak_deps=True
Last metadata expiration check: 0:41:19 ago on Fri 04 Jun 2021 12:30:05 PM CEST.
Dependencies resolved.
================================================================================
 Package                  Architecture Version               Repository    Size
================================================================================
Reinstalling:
 gawk                     x86_64       5.1.0-3.fc34          fedora       1.0 M
Installing weak dependencies:
 gawk-all-langpacks       x86_64       5.1.0-3.fc34          fedora       215 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 1.2 M
Installed size: 2.8 M
Is this ok [y/N]: no
Operation aborted.

Are supplements expected to be covered by the disfavor_unmet_weak_deps=True logic?

j-mracek · 2021-06-04T11:48:49Z

I investigated the issue and I found the problem. The problem is that the original package recommended `exiv2 = 0.27.3-5.fc34` therefore the logic disfavored `exiv2-0.27.3-5.fc34.x86_64@fedora` but not `exiv2-libs-0.27.3-7.fc34.x86_64@updates`. The new package `Recommends: exiv2 = 0.27.3-7.fc34`. I am not sure what we should do. What's your opinion?

adelton · 2021-06-04T12:08:13Z

That would make it one of the 196 cases mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1699672#c43. In https://bugzilla.redhat.com/show_bug.cgi?id=1699672#c44 Miro suggested that we could discourage using that form.

Still, maybe some heuristics could help -- for example if both those packages in question have the same source rpm and the version of the one carrying the Recommends matches the version in the Recommends clause, then the = %{version}-%{release} could be disregarded?

j-mracek · 2021-06-04T13:13:20Z

It is possible to add additional heuristic. But more conditions means more resources for resolvement. Each additional condition means potential new set of problems and complications for debugging.

j-mracek · 2021-06-04T13:26:18Z

Still, maybe some heuristics could help -- for example if both those packages in question have the same source rpm and the version of the one carrying the Recommends matches the version in the Recommends clause, then the = %{version}-%{release} could be disregarded?

I was also thinking about additional logic. I don't like to use source rpm name because any query that uses it is quite slow. I was thinking about logic like - if weakdep is unsatisfied and it is versioned add all packages that provides weak name. It will resolve the issue but I am not sure whether it is a good idea. Ok the logic will be even more complicated to not jump into another edge case.

j-mracek · 2021-06-04T14:40:22Z

@adelton I created a patches that changed the logic . It also requires a new version of libdnf.

j-mracek · 2021-06-10T09:35:58Z

It looks like that disfavors brake handling of debug-source packages. The behaviour is covered by test plugins-core/debuginfo-install.feature in our CI.

@bz1532378
Scenario: debugsource is installed even if debuginfo is present on the system
Given I successfully execute dnf with args "install --setopt=install_weak_deps=False foo-debuginfo"
 When I execute dnf with args "debuginfo-install foo"
 Then the exit code is 0
 Then Transaction is following
      | Action        | Package                                   |
      | install       | foo-debugsource-0:2.0-1.x86_64            |

The problem is in disfavors order. Try sudo dnf install zvbi --setopt=disfavor=zvbi and dnf starts to prefer i686 architecture because it disfavors i686 first and then x86_64 and the last disfavor is stronger. The same problem is with versions. With disfavor dnf can start to install lower version as a primary candidate. Even the --best option will not help.

j-mracek · 2021-06-11T07:57:52Z

I changed the code and it should work correctly - proper best candidate, repo priority and preference of the architecture. May I ask for retest of the latest implementation?

adelton · 2021-06-14T15:16:12Z

I'm using dnf-4.7.1-1.git.9427.1a4b359.fc34.noarch and it works fine for me. But I did not specifically try to find scenarios to break it. :-)

It disfavor packages for solver and remove them from the list of candidates for weak deps. It will prevents of installing disfavored packages that are recommended or supplemented. https://bugzilla.redhat.com/show_bug.cgi?id=1699672

Versioned provides which differs between packages can be incorrectly detected by disfavor_unmet_weak_deps heuristic. It can be resolve by additional search with the reldep name.

The order of disfavors is important for the solver but not wanted for auto-detection where there is not predictable order of disfavors.

j-mracek · 2021-06-16T15:12:07Z

I discovered another problem with disfavors. Disfavors are stronger than Obsoletes.
Example:
installed packages
pkg-A-1-1.noarch that recommends pkg-C
pkg-B-1-1.noarch

Available packages
pkg-C-2-2.noarch that obsoletes pkg-B and provides pkg-B

Expected behaviour for upgrade: pkg-B is replaced by pkg-C. But pkg-C is disfavored by disfavor_unmet_weak_deps therefore the obsolete is ignored.

j-mracek force-pushed the weakexcludes2 branch from f6740e3 to 0c60438 Compare June 2, 2021 18:36

j-mracek force-pushed the weakexcludes2 branch from e5392fd to 4d51891 Compare June 7, 2021 07:54

j-mracek marked this pull request as ready for review June 10, 2021 08:52

j-mracek force-pushed the weakexcludes2 branch from 4d51891 to 69978f1 Compare June 10, 2021 10:42

j-mracek added the blocked label Jun 10, 2021

j-mracek added ready for review and removed blocked labels Jun 11, 2021

j-mracek added 4 commits June 15, 2021 09:18

Add disfavored packages to the goal (RhBug:1699672)

8503799

It disfavor packages for solver and remove them from the list of candidates for weak deps. It will prevents of installing disfavored packages that are recommended or supplemented. https://bugzilla.redhat.com/show_bug.cgi?id=1699672

Add additional heuristic to handle versioned weak deps

4e715f6

Versioned provides which differs between packages can be incorrectly detected by disfavor_unmet_weak_deps heuristic. It can be resolve by additional search with the reldep name.

Change order of application of disfavors

19c319a

Disfavor unmet weak deps with the same weight

182fd9b

The order of disfavors is important for the solver but not wanted for auto-detection where there is not predictable order of disfavors.

j-mracek force-pushed the weakexcludes2 branch from 1a4b359 to 182fd9b Compare June 15, 2021 07:18

j-mracek added blocked and removed ready for review labels Jun 17, 2021

j-mracek mentioned this pull request Jun 18, 2021

Cannot use disfavor to not install packages as weak deps. openSUSE/libsolv#452

Closed

j-mracek mentioned this pull request Jun 23, 2021

Add weak_excluded packages to the goal (RhBug:1699672) #1780

Merged

j-mracek closed this Jun 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add disfavored packages to the goal (RhBug:1699672) #1774

Add disfavored packages to the goal (RhBug:1699672) #1774

j-mracek commented Jun 2, 2021 •

edited

j-mracek commented Jun 4, 2021

adelton commented Jun 4, 2021

adelton commented Jun 4, 2021 •

edited

j-mracek commented Jun 4, 2021 •

edited

adelton commented Jun 4, 2021

adelton commented Jun 4, 2021

j-mracek commented Jun 4, 2021 via email •

edited

adelton commented Jun 4, 2021 •

edited

j-mracek commented Jun 4, 2021

j-mracek commented Jun 4, 2021

j-mracek commented Jun 4, 2021 •

edited

j-mracek commented Jun 10, 2021 •

edited

j-mracek commented Jun 11, 2021

adelton commented Jun 14, 2021

j-mracek commented Jun 16, 2021

Add disfavored packages to the goal (RhBug:1699672) #1774

Add disfavored packages to the goal (RhBug:1699672) #1774

Conversation

j-mracek commented Jun 2, 2021 • edited

j-mracek commented Jun 4, 2021

Set the rule only for the transaction

Set the rule permanently in /etd/dnf/dnf.conf

Set the rule permanently in /etd/dnf/dnf.conf using config-manager

adelton commented Jun 4, 2021

adelton commented Jun 4, 2021 • edited

j-mracek commented Jun 4, 2021 • edited

adelton commented Jun 4, 2021

adelton commented Jun 4, 2021

j-mracek commented Jun 4, 2021 via email • edited

adelton commented Jun 4, 2021 • edited

j-mracek commented Jun 4, 2021

j-mracek commented Jun 4, 2021

j-mracek commented Jun 4, 2021 • edited

j-mracek commented Jun 10, 2021 • edited

j-mracek commented Jun 11, 2021

adelton commented Jun 14, 2021

j-mracek commented Jun 16, 2021

j-mracek commented Jun 2, 2021 •

edited

Set the rule permanently in `/etd/dnf/dnf.conf`

Set the rule permanently in `/etd/dnf/dnf.conf` using `config-manager`

adelton commented Jun 4, 2021 •

edited

j-mracek commented Jun 4, 2021 •

edited

j-mracek commented Jun 4, 2021 via email •

edited

adelton commented Jun 4, 2021 •

edited

j-mracek commented Jun 4, 2021 •

edited

j-mracek commented Jun 10, 2021 •

edited