Pep-503, implementation questions.

[Carreau]

Hi there,

I'm in the process of implementing pep-503 : "Simple Repository API", from @dstufft , in particular the data-requires-python attribute that recently landed in pip.

It's working, I just need to clean things up.

I'm going through the pep itself, and have a few questions, in particular while I'm in the codebase, I wanted to comply more to the pep-503.

Within a repository, the root URL ( / ) MUST be a valid HTML5 page

Thought the current version of pypi/simple includes <meta name="api-version" value="2" /> which is not valid HTML5. Is it supposed to be removed ? Or is this a historical backward compatibility reason ?

The pep makes no mentions of this meta tag and attribute, should it be fixed if we keep the tag ?

The links for each projects files have rel='internal' which is not valid either. Should it be removed ? If not what is it's reason to be ? If such a reason exist, should it be added to the pep?

The implementation of data-requires needs a SQL join, I came up with :

select filename, releases.requires_python, md5_digest
from release_files
inner join releases
    on release_files.version=releases.version 
    and release_files.name=releases.name
where release_files.name=%s

Are the SQL queries performance critical, or should it be fine because everything is cachd at CDN level ?

Some table have a python_version, I have no clue what this could be, it's queried, in the functions building the links, but not actually being used to build the links themselves.

What's the policy for "backward compatibility" of functions, that is to say if a function is used only in one place in legacy-pypi can I change its signature/returns value in an incompatible manner ?

Also curious about how frequently PyPI-legacy is actually deployed on the main PyPI instance, or test-pypi.

Otherwise, thanks a lot for all your hard work on PyPI-legacy/warehouse, now that I have a local copy working I might be to give a hand every now and then.

[dstufft]

Within a repository, the root URL ( / ) MUST be a valid HTML5 page

Thought the current version of pypi/simple includes which is not valid HTML5. Is it supposed to be removed ? Or is this a historical backward compatibility reason ?

This is a legacy thing yea, it can probably be removed at this point.

The links for each projects files have rel='internal' which is not valid either. Should it be removed ? If not what is it's reason to be ? If such a reason exist, should it be added to the pep?

Same thing as the api-version thing. There was an interim PEP before PEP 503 that used them, but PEP 470 killed them.

Are the SQL queries performance critical, or should it be fine because everything is cachd at CDN level ?

It's not the end of the world if the query isn't optimal, we cache these in Fastly for a long time, that being said it also shouldn't take 30 seconds or something like that to execute the query.

Some table have a python_version, I have no clue what this could be, it's queried, in the functions building the links, but not actually being used to build the links themselves.

Files have an associated python_version, it used to be used to generate the URL but it is no longer used for that. It being queried is likely just an artifact of it's old role and is probably not needed.

What's the policy for "backward compatibility" of functions, that is to say if a function is used only in one place in legacy-pypi can I change its signature/returns value in an incompatible manner ?

Assuming you mean the functions in PyPI and not the functions we expose say, via XMLRPC or so, there's no policy of compatibility. We can change whatever we want whenever we want. The only real caveat to that is there's basically no test suite, so we try not to change much if we can help it, just because it tends to mean less breakage in general.

Also curious about how frequently PyPI-legacy is actually deployed on the main PyPI instance, or test-pypi.

They both get auto deployed every uh, 15 minutes or so, testpypi gets deployed from the master branch, and PyPI itself gets deployed from the production branch.

[Carreau]

This is a legacy thing yea, it can probably be removed at this point.

Will do !

Same thing as the api-version thing. There was an interim PEP before PEP 503 that used them, but PEP 470 killed them.

Ah Yes ! THanks ! Will kill these as well !

It's not the end of the world if the query isn't optimal, we cache these in Fastly for a long time, that being said it also shouldn't take 30 seconds or something like that to execute the query.

Ok, that's should be fine, the time to make the query in only a few ms on my machine but still close to 2x what is was before, so I was wondering.

Assuming you mean the functions in PyPI and not the functions we expose say, via XMLRPC or so

Yes, that was what I meant.

The only real caveat to that is there's basically no test suite, so we try not to change much if we can help it, just because it tends to mean less breakage in general.

Understood.

They both get auto deployed every uh, 15 minutes or so, testpypi gets deployed from the master branch, and PyPI itself gets deployed from the production branch.

Good to know, all these informations are helpful ! I'll make a PR accordingly.

Thanks!

[ewdurbin]

@Carreau now 503 is implemented in pypi-legacy, closing!