Skip to content

Commit

Permalink
Add journal usage guidance, remove download methods in API docs (pypi…
Browse files Browse the repository at this point in the history 
…#3173)

* Add security and feed information to docs

Ref. pypi#1323

* Update XML-RPC API docs fields, usage info

After testing, update fields and briefly describe responses.

* Update API docs since PyPI no longer offers download stats

* Add API link to user help

Ref: pypi#2413.
  • Loading branch information
@brainwane @Mariatta
brainwane authored and Mariatta committed Mar 9, 2018
1 parent 44241e2 commit 547ef1a
Show file tree
Hide file tree
Showing 5 changed files with 103 additions and 30 deletions.
52 changes: 51 additions & 1 deletion docs/api-reference/feeds.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Feeds
=====

PyPI offers two RSS feeds, the `Newest Packages Feed`_ and the `Latest Updates
Feed`_.
Feed`_. You can also call its APIs to get more details on project activity.


Newest Packages Feed
Expand All @@ -19,3 +19,53 @@ Latest Updates Feed
Available at https://pypi.org/rss/updates.xml, this feed provides the latest
newly created releases for individual projects on PyPI, including the project
name and description, release version, and a link to the release page.

Project and release activity details
------------------------------------

PyPI publishes a "journal" of all project, package, and release
activity (including Owner and Maintainer additions and removals, and
source file and release additions and removals). You can query it with
a mix of :ref:`changelog-since` and :ref:`simple-api`. Call
``changelog_last_serial()`` (in :doc:`xml-rpc`) to get the current
revision of the journal (the last event's serial ID), then look at
``/simple/`` to get a list of all packages that currently
exist. Subsequently, you can call
``changelog_since_serial(since_serial)`` with the serial ID you
retrieved, and get the list of all actions that have occurred since
then.

Example usage::

>>> import xmlrpc.client
>>> import arrow
>>> client = xmlrpc.client.ServerProxy('https://test.pypi.org/pypi')
>>> latefeb = arrow.get('2018-02-20 10:00:00')
>>> latefeb.timestamp
1519120800
>>> latefebstamp = latefeb.timestamp
>>> recentchanges = client.changelog(latefebstamp)
>>> len(recentchanges)
7322
>>> for entry in recentchanges:
... if entry[0] == 'twine':
... print(entry[1], " ", entry[3], " ", entry[2])
...
...
...
None add Owner brainwane 1519952529
1.10.0 add py2.py3 file twine-1.10.0-py2.py3-none-any.whl 1520023899
1.10.0 new release 1520023899
1.10.0rc1 add py2.py3 file twine-1.10.0rc1-py2.py3-none-any.whl 1520023900
1.10.0rc1 new release 1520023900
1.10.0rc1 add source file twine-1.10.0rc1.tar.gz 1520023902
1.10.0 add source file twine-1.10.0.tar.gz 1520023903
1.10.0 remove file twine-1.10.0.tar.gz 1520024758
1.10.0 remove file twine-1.10.0-py2.py3-none-any.whl 1520024797
1.10.0 remove 1520025270


You could also request ``GET /simple/``, and record the ``ETag``, and
then periodically do a conditional HTTP GET to ``/simple/`` with that
ETag included. A 200 OK response indicates something has been added or
removed; if you get a 304 Not Modified, then nothing has changed.
1 change: 1 addition & 0 deletions docs/api-reference/legacy.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ Legacy API
The "Legacy API" provides feature parity with `pypi-legacy`_, hence the term
"legacy".

.. _simple-api:

Simple Project API
------------------
Expand Down
65 changes: 36 additions & 29 deletions docs/api-reference/xml-rpc.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ returned.
``release_data`` The `stable_version` flag is always an empty string. It was
never fully supported anyway.

``release_downloads`` and ``top_packages`` No longer supported. Please
use `Google BigQuery
<https://mail.python.org/pipermail/distutils-sig/2016-May/028986.html>`_
instead (`tips
<https://langui.sh/2016/12/09/data-driven-decisions/>`_).

Package Querying
----------------
Expand All @@ -62,53 +67,57 @@ Package Querying
Retrieve a list of `[role, package_name]` for a given `user`.
Role is either `Maintainer` or `Owner`.

``release_downloads(package_name, release_version)``
Retrieve a list of `[filename, download_count]` for a given `package_name`
and `release_version`.

``release_urls(package_name, release_version)``
Retrieve a list of download URLs for the given `release_version`.
Returns a list of dicts with the following keys:

* url
* packagetype ('sdist', 'bdist_wheel', etc)
* filename
* size
* md5_digest
* downloads
* has_sig
* packagetype ('sdist', 'bdist_wheel', etc)
* python_version (required version, or 'source', or 'any')
* size (an ``int``)
* md5_digest
* digests (a dict with two keys, "md5" and "sha256")
* has_sig (a boolean)
* upload_time (a ``DateTime`` object)
* comment_text
* downloads (always says "-1")
* url

``release_data(package_name, release_version)``
Retrieve metadata describing a specific `release_version`.
Returns a dict with keys for:

* name
* version
* stable_version (always an empty string)
* stable_version (always an empty string or None)
* bugtrack_url
* package_url
* release_url
* docs_url (URL of the packages.python.org docs if they've been supplied)
* home_page
* download_url
* project_url
* author
* author_email
* maintainer
* maintainer_email
* home_page
* license
* summary
* description
* description (string, sometimes the entirety of a ``README``)
* license
* keywords
* platform
* download_url
* classifiers (list of classifier strings)
* requires
* requires_dist
* provides
* provides_dist
* requires_external
* requires_python
* obsoletes
* obsoletes_dist
* project_url
* docs_url (URL of the packages.python.org docs if they've been supplied)
* requires_python
* requires_external
* _pypi_ordering
* _pypi_hidden
* downloads (``{'last_day': 0, 'last_week': 0, 'last_month': 0}``)

If the release does not exist, an empty dictionary is returned.

Expand Down Expand Up @@ -146,10 +155,6 @@ Package Querying
the given classifiers. `classifiers` must be a list of Trove classifier
strings.

``top_packages([number])``
Retrieve the sorted list of packages ranked by number of downloads.
Optionally limit the list to the `number` given.

``updated_releases(since)``
Retrieve a list of package releases made since the given timestamp. The
releases will be listed in descending release date.
Expand All @@ -159,22 +164,24 @@ Package Querying
since the given timestamp. The packages will be listed in descending date
of most recent change.

.. _changelog-since:

Mirroring Support
-----------------

``changelog(since, with_ids=False)``
Retrieve a list of `[name, version, timestamp, action]`, or
`[name, version, timestamp, action, id]` if `with_ids=True`, since the given
`since`. All `since` timestamps are UTC values. The argument is a UTC integer
seconds since the epoch.
Retrieve a list of `[name, version, timestamp, action]`, or `[name,
version, timestamp, action, id]` if `with_ids=True`, since the given
`since`. All `since` timestamps are UTC values. The argument is a
UTC integer seconds since the epoch (e.g., the ``timestamp`` method
to a ``datetime.datetime`` object).

``changelog_last_serial()``
Retrieve the last event's serial id.
Retrieve the last event's serial id (an ``int``).

``changelog_since_serial(since_serial)``
Retrieve a list of `(name, version, timestamp, action, serial)` since the
event identified by the given `since_serial` All timestamps are UTC
event identified by the given ``since_serial``. All timestamps are UTC
values. The argument is a UTC integer seconds since the epoch.

``list_packages_with_serial()``
Expand Down
7 changes: 7 additions & 0 deletions docs/security.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,14 @@
Security
========

Security policy
---------------
To read the most up to date version of our security policy, please visit
the application security page, available via the site_ footer.

Project and release activity details
------------------------------------
Please see :doc:`api-reference/feeds` for how to track new and updated
releases on PyPI.

.. _site: https://pypi.org
8 changes: 8 additions & 0 deletions warehouse/templates/pages/help.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
{% macro private_indices() %}How can I publish my private packages to PyPI?{% endmacro %}
{% macro admin_intervention() %}Why did my package or user registration get blocked?{% endmacro %}
{% macro file_name_reuse() %}Why am I getting "File already exists" error?{% endmacro %}
{% macro APIs() %}Does PyPI have APIs I can use?{% endmacro %}
{% block title %}Help{% endblock %}

{% block content %}
Expand All @@ -64,6 +65,7 @@ <h1 class="page-title">Common Questions</h1>
<li><a href="#private-indices">{{ private_indices() }}</a></li>
<li><a href="#admin-intervention">{{ admin_intervention() }}</a></li>
<li><a href="#file-name-reuse">{{ file_name_reuse() }}</a></li>
<li><a href="#APIs">{{ APIs() }}</a></li>
</ul>

<section id="packages" class="common-question">
Expand Down Expand Up @@ -285,6 +287,12 @@ <h2>{{ file_name_reuse() }}</h2>
To avoid this situation, <a href="https://packaging.python.org/guides/using-testpypi/">use Test PyPI to perform and check your upload first</a>, before uploading to <a href="https://pypi.org">pypi.org</a>.
</p>
</section>
<section id="APIs" class="common-question">
<h2>{{ APIs() }}</h2>
<p>
Yes, including RSS feeds of new packages and new releases. <a href="https://warehouse.readthedocs.io/api-reference/">Please see the API reference.</a>
</p>
</section>
</div>
</section>

Expand Down

0 comments on commit 547ef1a

Please sign in to comment.