Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Adds pypi_update_top_package_stats management command #1160

Merged
merged 11 commits into from
May 24, 2024

Conversation

jefftriplett
Copy link
Contributor

@jefftriplett jefftriplett commented May 24, 2024

Related to #1065 and #834

This pulls data for the top 8000 Python packages, which hits about 4 to 5% of our packages, which isn't great.

pypi_update_download_package_stats.py is a little clunky. It requires Seth's monthly pypi stats bundle. https://github.com/sethmlarson/pypi-data

I'm not sure if it's worth using the GitHub Release api to try and find it or if we should go another direction.

@jefftriplett jefftriplett marked this pull request as draft May 24, 2024 03:24
Copy link
Member

@saadmk11 saadmk11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only 4 to 5% hits is quite a small amount. :( It would have been great if there was a PyPI data collection for Django ecosystem specifically.

jefftriplett and others added 8 commits May 24, 2024 08:55
…ts.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
…ts.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
@jefftriplett
Copy link
Contributor Author

Only 4 to 5% hits is quite a small amount. :( It would have been great if there was a PyPI data collection for Django ecosystem specifically.

Agreed. That's why I added the other tool but it's kind of annoying if everything isn't automated. Definitely doable with a v2.

@jefftriplett jefftriplett marked this pull request as ready for review May 24, 2024 19:42
@jefftriplett
Copy link
Contributor Author

Since this is new and doesn't change anything that's existing, I'm going to go with it.

@jefftriplett jefftriplett merged commit c819471 into main May 24, 2024
3 checks passed
@jefftriplett jefftriplett deleted the adds-pypi-download-stats branch May 24, 2024 19:45
jefftriplett added a commit that referenced this pull request May 24, 2024
* ✨ Adds pypi_update_top_package_stats management command

* 🔥 Removes unused variable

* ✨ Adds other management command

* 🤝 Update package/management/commands/pypi_update_download_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* 🤝 Update package/management/commands/pypi_update_top_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 🤝 Update package/management/commands/pypi_update_top_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 🚜 Update package/management/commands/pypi_update_download_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 📝 Apply suggestions from code review

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 🔥 Removes filename

* 🚜 Applies PR feedback

---------

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
jefftriplett added a commit that referenced this pull request May 24, 2024
* 👕 Lints .env.local.example

* ⚙️ Adds compose to ignore list

* 🔥 Remove compose override example

* 👕 Lints start-dev.sh

* 🚜 Upgrades compose recipes

* 🚜 Update docker-compose and python -m usage

* 🚜 docker-compose => docker compose

* ✨ Adds pypi_update_top_package_stats management command (#1160)

* ✨ Adds pypi_update_top_package_stats management command

* 🔥 Removes unused variable

* ✨ Adds other management command

* 🤝 Update package/management/commands/pypi_update_download_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* 🤝 Update package/management/commands/pypi_update_top_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 🤝 Update package/management/commands/pypi_update_top_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 🚜 Update package/management/commands/pypi_update_download_package_stats.py

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 📝 Apply suggestions from code review

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>

* 🔥 Removes filename

* 🚜 Applies PR feedback

---------

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* 🔖 bump version 2024.5.6 -> 2024.5.7

* 🚜 docker-compose => docker compose

---------

Co-authored-by: Maksudul Haque <saad.mk112@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Automated Publisher <actions@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants