Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto publish and distribute #1084

Closed
wants to merge 1 commit into from

Conversation

dralley
Copy link
Contributor

@dralley dralley commented Jan 22, 2021

This is like 10% finished, I just want to discuss ideas

@pulpbot
Copy link
Member

pulpbot commented Jan 22, 2021

WARNING!!! This PR is not attached to an issue. In most cases this is not advisable. Please see our PR docs for more information about how to attach this PR to an issue.

@dralley dralley force-pushed the auto-publish-distribute branch 4 times, most recently from f6487fc to 679b00d Compare January 29, 2021 17:21
@daviddavis daviddavis marked this pull request as draft February 9, 2021 14:37
@dralley dralley force-pushed the auto-publish-distribute branch 8 times, most recently from f60b857 to 3cbb79b Compare February 16, 2021 00:02
dralley added a commit to dralley/pulp_file that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_rpm that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_file that referenced this pull request Feb 16, 2021
@dralley dralley force-pushed the auto-publish-distribute branch 2 times, most recently from 98ac024 to 3a9f98c Compare February 16, 2021 03:00
dralley added a commit to dralley/pulp_file that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_rpm that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_file that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_rpm that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_file that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_rpm that referenced this pull request Feb 16, 2021
dralley added a commit to dralley/pulp_container that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_container that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_npm that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_container that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_npm that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_deb that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_deb that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_container that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_container that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_python that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_ansible that referenced this pull request Feb 24, 2021
dralley added a commit to dralley/pulp_file that referenced this pull request Mar 4, 2021
dralley added a commit to dralley/pulp_rpm that referenced this pull request Mar 4, 2021
dralley added a commit to dralley/pulp_file that referenced this pull request Mar 4, 2021
dralley added a commit to dralley/pulp_container that referenced this pull request Mar 4, 2021
def create(self, validated_data):
"""Override the default create to use get_or_create behavior."""
instance, _ = self.Meta.model.objects.get_or_create(**validated_data)
return instance
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to disagreement on the way this works, maybe it's a step too far.

The idea here is that we now effectively have all the publish settings duplicated between *Publication and *PublishSettings objects, which is unfortunate. One way we could unify everything is by moving the data into PublishSettings exclusively, and attaching the PublishSettings to the Publication.

But that would also lead to a lot of duplicate PublishSettings objects unless we implement get_or_create() semantics, which is why I did this.

But if we just ignore the duplication of fields, it does avoid all of these changes, at the expense of the data model being a little less consistent.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify for me: with the code I see in this PR this does have everything live on PublishSettings exclusively yes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every plugin went and added fields to their own Publication models like this: https://github.com/pulp/pulp_rpm/blob/master/pulp_rpm/app/models/repository.py#L310-L325

This is how the user can know what settings were used in the creation of their publication.

Now that we have a PublishSettings model, all of those fields are duplicated in the plugins.

The question is, do we leave it like that, or do we try to move towards just creating a PublishSettings any time you publish (with get_or_create() semantics to avoid spamming them) and including that on the publication.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I misread your question. Most of the code related to this lives here, but under this framework one thing that would change is that the publish viewset would always get_or_create() a publish settings object and then pass it into the publish task instead of individual settings.

@@ -0,0 +1 @@
Added a new model `PublishSettings` which plugin writers are expected to subclass. When dispatching a task to create a `Publication`, use ".get_or_create()" to create a `PublishSettings` object and refer your publish tasks to use it. When creating a `Publication` inside your publish task with `Publication.create()`, you should pass the in the `PublishSettings` object so that it may be associated with the `Publication`. A new method `on_new_version()` should be added to repository models to define behavior that takes place when new repository versions are created. For an example of these changes, please see: https://github.com/pulp/pulp_file/pull/467
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this line-wrap at 100 chars please?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also sphinx needs double backticks instead of single for rst formatting

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This content is great. We also need this to go into a new section of docs in the plugin writer guide I think. Please add that too.

@dralley dralley force-pushed the auto-publish-distribute branch 2 times, most recently from fb439fa to 805551e Compare April 2, 2021 01:36
dralley added a commit to dralley/pulp_file that referenced this pull request Apr 2, 2021
dralley added a commit to dralley/pulp_file that referenced this pull request Apr 2, 2021
@dralley dralley closed this Apr 2, 2021
fao89 pushed a commit to pulp/pulp_npm that referenced this pull request May 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants