Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python package distribution contents exceed PyPI size limit #76

Open
lbianchi-lbl opened this issue Sep 22, 2023 · 1 comment
Open

Python package distribution contents exceed PyPI size limit #76

lbianchi-lbl opened this issue Sep 22, 2023 · 1 comment
Assignees
Labels
Priority:Normal Normal Priority Issue or PR

Comments

@lbianchi-lbl
Copy link
Contributor

  • The standard packaging for release builds both sdist and bdist_wheel, which, combined, exceed the default maximum size that PyPI allows (2 x ~73M > 100M)
  • Most of the space seems to be occupied by the surrogate notebooks (my guess is because of the multiple plots embedded as base-64-encoded PNG images):
$ find . -type f -exec du -h {} + | sort -h -r
73M	./.git/objects/pack/pack-8fa624c01e577f39a536768a80f06c63b74ec183.pack
7.7M	./idaes_examples/archive/power_gen/ngfc/NGFC_flowsheet_solution.json.gz
7.7M	./idaes_examples/archive/power_gen/ngfc/NGFC_flowsheet_init.json.gz
6.8M	./idaes_examples/archive/power_gen/ngfc/kriging_coefficients.dat
3.9M	./idaes_examples/notebooks/docs/surrogates/omlt/keras_flowsheet_optimization_usr.ipynb
3.9M	./idaes_examples/notebooks/docs/surrogates/omlt/keras_flowsheet_optimization_test.ipynb
3.9M	./idaes_examples/notebooks/docs/surrogates/omlt/keras_flowsheet_optimization.ipynb
3.9M	./idaes_examples/notebooks/docs/surrogates/omlt/keras_flowsheet_optimization_doc.ipynb
3.9M	./idaes_examples/notebooks/docs/surrogates/alamo/alamo_flowsheet_optimization_usr.ipynb
3.9M	./idaes_examples/notebooks/docs/surrogates/alamo/alamo_flowsheet_optimization_test.ipynb
3.9M	./idaes_examples/notebooks/docs/surrogates/alamo/alamo_flowsheet_optimization.ipynb
3.9M	./idaes_examples/notebooks/docs/surrogates/alamo/alamo_flowsheet_optimization_doc.ipynb
3.8M	./idaes_examples/notebooks/docs/surrogates/pysmo/pysmo_flowsheet_optimization_usr.ipynb
3.8M	./idaes_examples/notebooks/docs/surrogates/pysmo/pysmo_flowsheet_optimization_test.ipynb
3.8M	./idaes_examples/notebooks/docs/surrogates/pysmo/pysmo_flowsheet_optimization.ipynb
3.8M	./idaes_examples/notebooks/docs/surrogates/pysmo/pysmo_flowsheet_optimization_doc.ipynb
3.3M	./idaes_examples/notebooks/docs/surrogates/pysmo_krig_surrogate.json
3.2M	./idaes_examples/notebooks/logo.png
3.2M	./idaes_examples/notebooks/docs/tut/ui/idaes-logo.png
3.2M	./idaes_examples/notebooks/_dev/notebooks/logo.png
1.4M	./idaes_examples/archive/power_gen/sofc/sofc_rom_data.json
1.4M	./idaes_examples/archive/power_gen/sofc/sofc_init.json.gz
1.1M	./idaes_examples/notebooks/docs/tut/ui/fv1.png
1.1M	./idaes_examples/archive/power_gen/supercritical/supercritical_power_plant_src.ipynb
992K	./idaes_examples/archive/data_reconciliation/data_plot_book.pdf
940K	./idaes_examples/notebooks/docs/surrogates/best_practices_optimization_usr.ipynb
940K	./idaes_examples/notebooks/docs/surrogates/best_practices_optimization_test.ipynb
940K	./idaes_examples/notebooks/docs/surrogates/best_practices_optimization_doc.ipynb
920K	./idaes_examples/notebooks/docs/surrogates/best_practices_optimization.ipynb
892K	./idaes_examples/notebooks/docs/surrogates/pysmo/data_files/matyas_function.xls
808K	./idaes_examples/notebooks/docs/surrogates/pysmo/data_files/three_humpback_data_v4.csv
808K	./idaes_examples/notebooks/docs/surrogates/pysmo/data_files/six_hump_function_data.tab
760K	./idaes_examples/notebooks/docs/surrogates/pysmo/data_files/griewank_data.txt
@lbianchi-lbl lbianchi-lbl self-assigned this Sep 22, 2023
@ksbeattie ksbeattie added the Priority:High High Priority Issue or PR label Sep 28, 2023
@ksbeattie ksbeattie added Priority:Normal Normal Priority Issue or PR and removed Priority:High High Priority Issue or PR labels Nov 9, 2023
@lbianchi-lbl
Copy link
Contributor Author

lbianchi-lbl commented Nov 9, 2023

Plan to address this:

  • Modify packaging configuration to filter out files when building sdist and bdist_wheel (e.g. exclude multiple suffixes for the same notebook)
  • Add CI check to ensure that new additions don't push the distribution payload to exceed the PyPI size limit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority:Normal Normal Priority Issue or PR
Projects
None yet
Development

No branches or pull requests

5 participants