Fixes #25452 - Duplicate repo in CCV causes mismatch

[johnpmitsch]

A composite content view containing two content views will only use
one filter for the repo and cause a mismatch between the published
repo on the filesystem and what pulp/katello displays.

[theforeman-bot]

Issues: #25452

[johnpmitsch]

To test:

• Create content view CV1 with 'zoo' repo (or repo of your choice)
• Create an exclude filter for a package (i.e. walrus) on zoo repo in CV1
• Create CV2 with same 'zoo' repo
• Create an exclude filter for a different package (i.e. camel) on zoo repo in CV2
• Publish both content views
• Create composite content view, add latest content view version for CV1 and CV2
• Check katello lists the full amount of packages for the repo (i.e. 32 for zoo)
  • Exclude filters are ignored when publishing CCV's with duplicate filters.
• Check pulp (using pulp-admin) lists full amount of packages.
• Check filesystem has full amount of packages. you can do this by going to /var/lib/pulp/published/yum/master/yum_distributor/<OrgID>-<CCVName>-Library-<repo-uuid>/<some-id>/Packages and then running ls -al | grep 'rpm' | wc -l. In master this should show package filtered with one of the CV filters (i.e. 31 packages for zoo repo). With this PR, it should be the full amount of packages (i.e. 32 for zoo)

This commit doesn't add the ability to use exclude filter in composite content views for duplicate repos, but rather fixes a mismatch between the filesystem and repo metadata

[johnpmitsch]

Going to try and figure out a good way to write tests for this, but my initial thought it is going to be pretty hard to test a mismatch between the filesystem and what pulp tells us. It's more an order-of-operations fix.

[johnpmitsch]

I added a sentence to the warning on the publish page to better convey the expected behavior. I'm open to better wording:

[beav]

if the metadata generation gets skipped here, where does it happen later on?

[johnpmitsch]

In this step plan_action(Katello::Repository::MetadataGenerate, new_repository) if repositories.length > 1

https://github.com/Katello/katello/pull/7836/files#diff-516603408c98ec56e67c629e9de94587R28

[beav]

oops, didnt see that even though it was right on the screen 👓

it's correct that in the case of incremental, no metadata is regenerated? The skip_metadata has two checks, but the regen at the end only validates the latter check

[johnpmitsch]

hrmmm... good point. Do we do incremental versions for composite content views?

[beav]

I think so, at least based on the text for the param in the api:

Content View Version Ids to perform an incremental update on. May contain composites as well as one or more components to update.

[johnpmitsch]

I'm not sure if there is a better way to display the logic, this was the most readable way I could figure out, but I'm open to suggestions 📂

The original functionality should be working as is. Here are some simplified scenarios

# incremental publish and normal CV
2.5.1 :002 > incremental = true; repositories = ['foo']
 => ["foo"]
# skip metadata 
2.5.1 :003 > incremental || repositories.length > 1
 => true 

# incremental and CVV
2.5.1 :004 > incremental = true; repositories = ['foo', 'bar']
 => ["foo", "bar"] 
2.5.1 :005 > incremental || repositories.length > 1
 => true 

# normal publish and CVV
2.5.1 :006 > incremental = false; repositories = ['foo', 'bar']
 => ["foo", "bar"] 
2.5.1 :007 > incremental || repositories.length > 1
 => true 

# normal publish and CV
2.5.1 :008 > incremental = false; repositories = ['foo']
 => ["foo"] 
# here we don't skip metadata regenerating in the task
2.5.1 :009 > incremental || repositories.length > 1
 => false 

If any CV publish is incremental, we are still skipping metadata, the addition is we are skipping metadata if its a CVV with duplicate repos as well

[beav]

code looks ok and tested ok for me

[jlsherrill]

ACK from me, based on beav's testing

[johnpmitsch]

@beav I tested out incremental updates (using hammer) on both content views and composite content views without any issues. Feel free to test any additional scenarios.