Implement/Integrate statistics

[dstufft]

The new design has a pane for statistics, however we need to actually implement and integrate them.

Relevant statistics to show:

• Download counts
• General PyPI traffic

[nlhkabu]

This tab will be removed for launch (see #1310) - we should consider this a long term 'todo' :)

For reference, this is the design.

[brainwane]

Related: #3400 & #3434 on https://libraries.io/pypi/ and https://packaging.python.org/guides/analyzing-pypi-package-downloads/ . The right thing to do here, like in #991, might be to aid @andrew in getting download stats into libraries.io (librariesio/libraries.io#1916), then pull a selected stat or two into the display in Warehouse but still point people to libraries.io and the Google BigQuery dataset for more details.

cc @a-wakeel and @micurd who opened related issues.

[andrew]

Related issue on the Libraries.io repo about download stats, will be put back on the roadmap within a few months I expect: librariesio/libraries.io#53

[di]

pypi/legacy#254 says:

as a package owner/maintainer, it'd be fantastic if there was any way to get something more fine-grained than "downloads per month/week/day".

i recognize that privacy concerns that put some limitations on here, but a few things i think might still be doable, in priority order:

1/ what package install led to me being installed?

this one would be huge -- if i'm maintaining a package that's a common dependency for several packages, which ones are driving my installs? in particular, if i need to change/deprecate something, which dependencies am i most likely to break?

2/ breakdowns by OS/python version

the biggest win here would of course be py2 vs. py3 -- how many of my users have already switched? on top of that, OS information (even just linux/mac/win) would be good, esp. around testing priorities.

3/ support for user-provided data in the install request

this one's quite general, but opens up all kinds of uses. my motivation: i'd love to know (eg) all installs that come as part of a google-owned tool. these are different from a "how many people use my package" POV, and in this case, any nontrivial number probably points to somewhere we should be caching. but i can't see the breakdown, so i don't know.

[crflynn]

@di I currently support some of these breakdowns at pypistats.org; I can contribute here by migrating some functionality to warehouse with some guidance. Is there a detailed feature spec for displaying these metrics in warehouse?

[abitrolly]

I propose to split download stats and traffic stats into separate issues. This issue can serve as an Epic, but in its current form is it not actionable. An actionable issue about traffic stats consists of these steps.

• Find the source of traffic data
• Preprocess the data if needed
• Expose the data into API
• Design human interface

#10355 (comment) contains the ready to reuse API example and reporting from piwheels, and if not the practice of closing more specific issues in favor of more generic ones, it could be used as an actionable issue to gather more comments.