Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge colliding structure content #760

Merged
merged 2 commits into from Jun 19, 2023

Conversation

quba42
Copy link
Collaborator

@quba42 quba42 commented May 9, 2023

closes #599

This will require a DB migration that alters the structure content in existing repository versions.

The plan is to release this change as pulp_deb 3.0.0!

@quba42
Copy link
Collaborator Author

quba42 commented May 9, 2023

Important: This initial draft pull request adds the code changes with an automatically generated migration. This migration will fail for most existing data, since we need to fix any colliding content and all foreign keys pointing at it, before we can alter the uniqueness constraints.

@quba42 quba42 added .misc CHANGES/<issue_number>.misc .removal CHANGES/<issue_number>.removal labels May 10, 2023
@quba42 quba42 added the .bugfix CHANGES/<issue_number>.bugfix label May 22, 2023
@quba42 quba42 force-pushed the structure_problem_migration branch 3 times, most recently from df8d06f to 9ad7522 Compare May 23, 2023 13:46
@quba42
Copy link
Collaborator Author

quba42 commented May 23, 2023

We now have a working state for the DB migration. However, I have yet to introduce batching. Once this is done the migration requires performance testing with installations with large amounts of APT content.

@quba42 quba42 force-pushed the structure_problem_migration branch 2 times, most recently from a2fec73 to f5c6018 Compare June 5, 2023 13:42
@quba42 quba42 force-pushed the structure_problem_migration branch 4 times, most recently from 2711a69 to ba27a37 Compare June 12, 2023 13:15
@quba42 quba42 force-pushed the structure_problem_migration branch 3 times, most recently from c66d912 to 0243b8d Compare June 14, 2023 15:10
@quba42
Copy link
Collaborator Author

quba42 commented Jun 14, 2023

Ugh. Why are the tests now failing. These were already green before. It looks suspiciously like some change somewhere else broke this test.

@quba42
Copy link
Collaborator Author

quba42 commented Jun 14, 2023

This looks like a good candidate for what broke the tests: pulp/pulpcore#3904

@hstct
Copy link
Contributor

hstct commented Jun 14, 2023

I think it is because the hosted data fixtures got updated? I ran into the same problem with the converted publish tests last week this is what i did to fix it for now: https://github.com/pulp/pulp_deb/pull/776/files#diff-39a085d2db2d6234c049924d46d2c3bac7ef2029a532cb9aa0e09d3a4a91f282R443

But it still worked for the pre conversion tests. I made a note to talk about this with you :)

The essentials field is definitely in the pkg metadata though I checked. Maybe it is new because of equivs-build got updated somehow (which is likely since the author field had some changes as well)?

Either way the field doesn't seem to get published. It is a valid field though https://www.debian.org/doc/debian-policy/ch-controlfields.html#essential

closes pulp#599

Includes a DB migration that re-writes any colliding structure content,
thus altering existing repository versions. So re-written repo versions
will result in structurally identical publications, as compared to
before the change. The only difference is that post-migration
publications will lose any duplicate package paragraphs.

This change will be released as part of pulp_deb version 3.0.0!
@quba42 quba42 force-pushed the structure_problem_migration branch from 0243b8d to c2de7cc Compare June 19, 2023 11:44
@quba42 quba42 marked this pull request as ready for review June 19, 2023 13:24
@quba42 quba42 merged commit bf2a1af into pulp:main Jun 19, 2023
7 of 8 checks passed
@quba42 quba42 deleted the structure_problem_migration branch June 19, 2023 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
.bugfix CHANGES/<issue_number>.bugfix .misc CHANGES/<issue_number>.misc .removal CHANGES/<issue_number>.removal
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ReleaseComponent content with different codename or suite but equal distribution collides
3 participants