Document API policies #3333
Document API policies #3333
Conversation
docs/api-reference/index.rst
Outdated
| For periodically checking for new packages or updates to existing packages, | ||
| please use our RSS feeds. | ||
|
|
||
| If at all possible, it is recommended to use the JSON/RSS/Legacy APIs over |
There was a problem hiding this comment.
Stronger language, may be worth stating that XMLRPC will be going away as soon as we find a way to.
No new integrations should use the XMLRPC interfaces that PyPI provides, it is planned for deprecation.
docs/api-reference/index.rst
Outdated
| ~~~~~~~~~~~~~ | ||
|
|
||
| Due to the heavy caching and CDN use, there is currently no rate limiting of | ||
| PyPI APIs. |
There was a problem hiding this comment.
*At the edge. Requests that hit our backends (uploads, XMLRPC) may be rate limited (automatically or manually) if they are causing degradation of service.
There was a problem hiding this comment.
Additionally we should reserve the right to temporarily prohibit a client based on activity.
docs/api-reference/index.rst
Outdated
| All API requests also provide an ``ETag`` header. If you're making a lot of | ||
| repeated requests, please ensure your API consumer will respect this header to | ||
| determine whether to actually repeat a request or not. | ||
|
|
There was a problem hiding this comment.
Need exception for XMLRPC here. At this point it provides no information about if the response was cached internally.
docs/api-reference/index.rst
Outdated
|
|
||
| All API requests are cached. Requests to the JSON, RSS or Legacy APIs are | ||
| cached by our CDN provider. You can determine if you've hit the cache based on | ||
| the ``X-Cache`` and ``X-Cache-Hist`` headers in the response. |
There was a problem hiding this comment.
Works for me, though we may want to be a bit more explicit about our ability to prohibit a client at any layer.
We do have the ability to prohibit a client or IP at the edge, at the nginx layer, or internally via the pyramid app and have used this approach for mitigating accidental DoS in the past.
|
@ewdurbin Might it be preferable to be less explicit about how we can prohibit malicious clients? I think for the average user, it's enough to just know "we can drop the 🔨 on you". |
There was a problem hiding this comment.
I like having this on the API reference index page, and this feels solid overall.
Request: add a note to the top of xml-rpc.rst saying we're working on deprecating this API and pointing to our APIs/feeds tag or similar.
docs/api-reference/index.rst
Outdated
| * Set your consumer's ``User-Agent`` header to uniquely identify your requests | ||
| * Try not to make a lot of requests (thousands) in a short amount of time | ||
| (minutes). Generally PyPI can handle it, but it's preferred to make requests | ||
| in serial over a longer amount of time if possible. |
There was a problem hiding this comment.
I suggest we add "consider running your own organizational mirror instead/in addition" & point to bandersnatch.
There was a problem hiding this comment.
I'm not sure that's good advice for folks that need consume our APIs. Does bandersnatch provide all the exact same APIs as PyPI?
Generally I think "use bandersnatch" is good advice for users that either can't use pypi.org or are concerned about it's availability. I'm not sure that using it could be considered a "best practice" for consuming our API.
(With the exception of using a mirror for the Simple/download API, which could be helpful in reducing load on our CDN, but I think we don't really have the assumption that this is necessary or recommended for big consumers -- we just handle the traffic.)
There was a problem hiding this comment.
This document tells API users how to be good neighbors, so I suggest we add a line advising orgs that it would be friendly for them to use internal mirrors for Simple/downloads. But my opinion is mild here.
| degradation of service. | ||
|
|
||
| In addition, PyPI reserves the right to temporarily or permanently prohibit a | ||
| consumer based on irresponsible activity. |
There was a problem hiding this comment.
At a past org we suggested: use a descriptive User-Agent header with your contact info in it, so we can at least potentially ping you to suggest alternate approaches if you're causing a ruckus.
Fixes #2931.