Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Alternate Repository Location for PEP 708 #15716

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

cofiem
Copy link
Contributor

@cofiem cofiem commented Apr 3, 2024

Adds per-project Alternate Repository Location as per PEP 708.

  • Implements the UI and database changes.
  • Includes changes to the html and json APIs.
  • Includes tests.
  • The alternate repository location information is on the project settings page.
  • There is a form to add, and a list to show and delete existing entries.

Notes:

  • The 'tracks' metadata does not make sense for PyPI, as PyPI does not extend other repositories.

Relates to #15431

Previews:

HTML: curl http://localhost/simple/[project_name]/

<!DOCTYPE html>
<html>
  <head>
    <meta name="pypi:repository-version" content="1.1">
    <meta name="pypi:alternate-locations" content="https://example.org">
    <title>Links for [project_name]</title>
  </head>
...

JSON: curl -H "Accept: application/vnd.pypi.simple.v1+json" http://localhost/simple/[project_name]/

{"alternate-locations":["https://example.org"], ... }

image

@cofiem cofiem marked this pull request as ready for review June 30, 2024 05:32
@cofiem cofiem requested a review from a team as a code owner June 30, 2024 05:32
@ewdurbin
Copy link
Member

ewdurbin commented Jul 9, 2024

Hi @cofiem and thank you for your contribution! I'm going to ask @dstufft to take a look as PEP author.

Copy link
Member

@ewdurbin ewdurbin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really strong and comprehensive start, thank you! I left some notes after an initial review.

@@ -943,3 +948,34 @@ class ProjectMacaroonWarningAssociation(db.Model):
ForeignKey("projects.id", onupdate="CASCADE", ondelete="CASCADE"),
primary_key=True,
)


class AlternateRepository(db.Model):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are going to need to register a cache/purge keys for this model similar to "Release" to ensure that when AlternateRepository objects are created/destroyed we purge the relevant pages.

Our cache keys are registered here.

And my best guess is that we need to add

    config.register_origin_cache_keys(
        AlternateRepository,
        cache_keys=["project/{obj.project.normalized_name}"],
        purge_keys=[
            key_factory("project/{obj.project.normalized_name}"),
        ],
    )

To ensure that the simple index for the project is purged on update, to pick up the new metadata for display.

@@ -15,6 +15,10 @@
<html>
<head>
<meta name="pypi:repository-version" content="{{ meta['api-version'] }}">
{# <meta name="pypi:alternate-locations" content="{{ request.route_path('api.simple.detail', name=name|canonicalize_name) }}">#}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the PEP, I wasn't sure if the part about "... clients MUST implicitly assume that the url the response was fetched from was included in the list..." meant that the url of PyPI should be in the list presented to clients or not. If the url the client used, and the url rendered by PyPI differ, that might be a problem? Likely a question for @dstufft .

from warehouse.utils.cors import _CORS_HEADERS

MIME_TEXT_HTML = "text/html"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now have a constants module. Since these constants are not used elsewhere, I think we're fine to leave them in warehouse.api.simple, but thought I'd flag it if we are using them anywhere else.

@cofiem
Copy link
Contributor Author

cofiem commented Jul 10, 2024

Thanks very much for the initial review, @ewdurbin! I'll have a look at the cache keys for the model.

@@ -18,10 +18,11 @@
from pyramid.testing import DummyRequest

from warehouse.api import simple
from warehouse.packaging.utils import API_VERSION
from warehouse.packaging.utils import API_VERSION, _valid_simple_detail_context
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't changed the API_VERSION to 1.2, as I wasn't sure if or how PyPI presents different versions of APIs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants