Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How large is a full pypi mirror? #50

Closed
wayneworkman opened this issue Jul 12, 2018 · 2 comments
Closed

How large is a full pypi mirror? #50

wayneworkman opened this issue Jul 12, 2018 · 2 comments
Assignees

Comments

@wayneworkman
Copy link

@wayneworkman wayneworkman commented Jul 12, 2018

Just wondering. I'm at nearly 600GB right now - wondering how far there is to go.
Please date your responses as people in the future may come here for answers.

@cooperlees cooperlees self-assigned this Jul 12, 2018
@cooperlees

This comment has been minimized.

Copy link
Contributor

@cooperlees cooperlees commented Jul 12, 2018

I feel this is a misuse of issues, unless others see value in this I would like to propose to close this.

If you really want an accurate measurement of the size of total PyPI data footprint, please ask the Warehouse team to provide it somehow in real time (or close to).

They might already via Google Big Query if the file size is available:
https://bigquery.cloud.google.com/dataset/the-psf:pypi

  • From my quick look it only seems to show downloading of packages.

Anyone else know a way to get this data?

@cooperlees

This comment has been minimized.

Copy link
Contributor

@cooperlees cooperlees commented Jul 12, 2018

https://p.datadoghq.com/sb/7dc8b3250-85dcf667bd - Shows we're currenty at 2.04TB. You've got a ways to go.

@dstufft has opened pypa/warehouse#4288 to show who the top N users are space wise so we can formulate a good blacklist to space people tons of space.

@cooperlees cooperlees closed this Jul 13, 2018
@cooperlees cooperlees changed the title [QUESTION] How large is a full pypi mirror? How large is a full pypi mirror? Jul 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.