Use --exodus-commit=phase1 in pulp hook [RHELDST-20490] #28
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When this hook is active, the control flow is: 1. pubtools-exodus: create publish 2. rhsm-pulp: add content to publish 3. pubtools-exodus: commit publish Problem: step 2 might succeed from Pulp's point of view, but then fail at step 3 leaving the content not actually published to the CDN storage. As Pulp believes the publishes were successful, it would then skip publish of certain files at the next attempt, leading to missing content. The newly introduced concept of a phase1 commit should fix this by allowing step 2 to request that all "phase 1 content" (e.g. RPMs, but not repodata) are flushed to the CDN storage before proceeding. Go ahead and start using that via the new related exodus-rsync argument. While it would appear to make sense to do this at *every* publish, an env var is added as an escape hatch to go back to the old behavior, just in case something doesn't work as expected or upgrades are performed in the wrong order.
negillett
approved these changes
Oct 3, 2023
crungehottman
approved these changes
Oct 4, 2023
|
Holding back on submit of this until the latest exodus-rsync is deployed everywhere. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When this hook is active, the control flow is:
Problem: step 2 might succeed from Pulp's point of view, but then fail at step 3 leaving the content not actually published to the CDN storage. As Pulp believes the publishes were successful, it would then skip publish of certain files at the next attempt, leading to missing content.
The newly introduced concept of a phase1 commit should fix this by allowing step 2 to request that all "phase 1 content" (e.g. RPMs, but not repodata) are flushed to the CDN storage before proceeding. Go ahead and start using that via the new related exodus-rsync argument.
While it would appear to make sense to do this at every publish, an env var is added as an escape hatch to go back to the old behavior, just in case something doesn't work as expected or upgrades are performed in the wrong order.