Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server is down #53

Closed
cclauss opened this issue Apr 21, 2018 · 9 comments
Closed

Server is down #53

cclauss opened this issue Apr 21, 2018 · 9 comments

Comments

@cclauss
Copy link
Contributor

@cclauss cclauss commented Apr 21, 2018

http://python3wos.appspot.com

@ubershmekel
Copy link
Owner

@ubershmekel ubershmekel commented Apr 22, 2018

Thanks for letting me know. App engine dashboard mentions this since 7 days ago:

Fault: <Fault -32500: 'RuntimeError: This API has been removed. Please Use BigQuery instead.'>
at close (/base/alloc/tmpfs/dynamic_runtimes/python27/c5586dbb532f7e5f_unzipped/python27_dist/lib/python2.7/xmlrpclib.py:800)
at __parse_response (/base/data/home/apps/s~python3wos2/1.405580611997320593/pypi_parser.py:107)
at request (/base/data/home/apps/s~python3wos2/1.405580611997320593/pypi_parser.py:100)
at __request (/base/alloc/tmpfs/dynamic_runtimes/python27/c5586dbb532f7e5f_unzipped/python27_dist/lib/python2.7/xmlrpclib.py:1602)
at __call__ (/base/alloc/tmpfs/dynamic_runtimes/python27/c5586dbb532f7e5f_unzipped/python27_dist/lib/python2.7/xmlrpclib.py:1243)
at get_packages (/base/data/home/apps/s~python3wos2/1.405580611997320593/pypi_parser.py:160)
at fetch_and_cache_package_info (/base/data/home/apps/s~python3wos2/1.405580611997320593/pypi_cron.py:31)
at get_packages_list_from_cache_or_pypi (/base/data/home/apps/s~python3wos2/1.405580611997320593/pypi_cron.py:39)
at build_data (/base/data/home/apps/s~python3wos2/1.405580611997320593/main.py:13)
at get_html (/base/data/home/apps/s~python3wos2/1.405580611997320593/main.py:45)
at get (/base/data/home/apps/s~python3wos2/1.405580611997320593/main.py:59)

Also I see PyPI api failures since Feb 16th.

I might just archive the recent snapshot and be done with it.

@cclauss
Copy link
Contributor Author

@cclauss cclauss commented Apr 22, 2018

Understood...

All that remained red was Supervisor that is near to their Python 3 release and the Moz tools.

Not bad given that there are 619 days until Python 2 end of life.

from datetime import date ; print((date(2020, 1, 1) - date.today()).days,
                                  'days until Python 2 end of life.')

@edmorley @hugovk

@ubershmekel
Copy link
Owner

@ubershmekel ubershmekel commented Apr 22, 2018

We're back online. Again thank you @cclauss for letting me know.

This is the offending line, PyPI deprecated the function: package_name_downloads = CLIENT.top_packages(HOW_MANY_TO_CHART)

I took some recent download numbers from Big Query https://bigquery.cloud.google.com/table/the-psf:pypi.downloads20180422 using this query:

SELECT file.project, COUNT(*) as download_count
FROM TABLE_DATE_RANGE(
  [the-psf:pypi.downloads],
  TIMESTAMP("2018-04-01"),
  CURRENT_TIMESTAMP()
)
GROUP BY file.project
ORDER BY download_count DESC
LIMIT 200

I then downloaded the JSON, ran the rest of the code using the above data, got the template data that would have been used to render html, saved that, and from now on we're forever using that stale snapshot of data. I added some deprecation and stale data warning to the top of the site.

ubershmekel added a commit that referenced this issue Apr 22, 2018
Saving the template data to a file
I built this final data set from Google Big Query
More info at #53
@hugovk
Copy link
Contributor

@hugovk hugovk commented Apr 22, 2018

For fresh data, you could do something like meshy/pythonwheels#100.

@hugovk
Copy link
Contributor

@hugovk hugovk commented Apr 22, 2018

... but I think it's fine to say "mission over" and keep it as is.

Thank you very much for making this in the first place and helping to get to Python 3!

@brainwane
Copy link

@brainwane brainwane commented Apr 22, 2018

I wonder whether the libraries.io API https://libraries.io/api would be a useful substitute?

@cclauss cclauss closed this Apr 23, 2018
@cclauss
Copy link
Contributor Author

@cclauss cclauss commented May 11, 2018

@ubershmekel Can you please regenerate the page? I believe there has been some improvement.

@dmsquires
Copy link

@dmsquires dmsquires commented May 22, 2018

I would rather create a "wall of shame" for still-popular modules who haven't adopted a proper python3 path versus starting this up again.

Great job all around on this, I visited this site monthly for ages. It solidified my push to use python3 exclusively in all my projects a long time ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants