Minimize conda environment

[tobiasdiez]

The dev-conda environment contains quite a lot of packages that are not directly needed to compile or run sage, but only to compile one of sage's dependencies. Since conda already takes care of building the dependency tree (and only including runtime dependencies in it), there is no need to include the deep-dependencies in the environment file. Doing so just unnecessary bloats up the install time and size.

In fact, du -hs /opt/conda/envs/** shows

6.5G    /opt/conda/envs/sage-dev
1.4G    /opt/conda/envs/sage-dev-min

that the sage-dev environment is very large and there is a lot of room for improvement. The second env in the above list corresponds to the following env:

name: sage-dev-min
channels:
  - conda-forge
  - nodefaults
dependencies:
  - tar
  - cypari2 # sagelib/dep
  - singular # sagelib/dep
  - gap-defaults # sagelib/dep
# - html5lib # not referenced in src/
  - libgd # sagelib/dep
  - sagemath-db-polytopes
  - gengetopt
  - symmetrica # sagelib/dep
  - palp
# - sphinxcontrib-websupport # not referenced in src/
  - pkgconfig # sagelib/dep
  - git # Additional dev tool
# - pybind11 # not referenced in src/
  - cliquer # sagelib/dep
  - libatomic_ops
  - pari-elldata
  - zn_poly # sagelib/dep
  - pycodestyle # Additional dev tool
  - pari-galpol
  - gfan # features/gfan
  - memory-allocator # sagelib/dep
  - prompt_toolkit
  - eclib # sagelib/dep
  - arb # sagelib/dep
  - tachyon # sagemath_doc_html/dep
  - bc
  - threejs-sage
  - lcalc # sagelib/dep
  - sagemath-db-graphs
# - ninja # not referenced in src/
  - backports.zoneinfo
# - cppy # not referenced in src/
  - networkx
  - jupyter-jsmol
  - pcre
  - sympy
# - appnope # not referenced in src/
  - flintqs # interfaces/qsieve but not declared as dep
  - r
  - matplotlib
  - r-essentials
  - cvxopt # sagemath-standard/req
  - libhomfly # sagelib/dep
  - pari-seadata
# - poetry-core # not referenced in src/
  - linbox # sagelib/dep
# - importlib_metadata # mentioned in the docs but not used?
  - scipy # sagemath-standard/req
  - patch
# - qhull # not referenced in src/
  - jupyter-packaging
  - jupyter_sphinx # sagemath_doc_html/dependencies
  - compilers
  - esbonio # Additional dev tool
  - openssh # Additional dev tool
  - giac # sagelib/dep
  - fpylll # sagemath-standard/req
  - pplpy # sagelib/dep
# - hatchling # not referenced in src/
  - sagemath-db-elliptic-curves
  - jmol # sagemath_doc_html/dependencies
  - tox # additional dev tool but not declared as such
# - sagetex # not actively referenced in src/
  - primecountpy # sagelib/dep
  - furo # sagemath_doc_html/dependencies
  - rpy2 # sagemath-standard/req
# - pythran # not referenced in src/
  - planarity # sagelib/dep
  - python-lrcalc # sagelib/dep
# - vcversioner # not referenced in src/
  - rw # sagelib/dep
  - brial # sagelib/dep
  - maxima
  - sympow # sage/lfunctions/sympow.py but not declared as dep
  - automake # Package needed for ./bootstrap
  - sagemath-db-combinatorial-designs
# - importlib-resources # not referenced in src/
# - cmake # only mentioned in docs as not often used
# - setuptools_scm # not referenced in src/
  - boost-cpp # sagelib/dep
  - blas # sagelib/dep
  - libbraiding # sagelib/dep
  - mathjax # sagemath_doc_html/dependencies
  - sagemath-db-conway-polynomials

This env has been created using https://github.com/conda-incubator/conda-tree minimizing function (which only takes the leaves of the dependency tree) and then manually going through the remaining dependencies to see which ones are not needed by sage (see comments). Sage compiles within this env, but I've not yet run the doctests. I propose to include this env file directly, replacing the existing automatically generated sage-dev.

Comments about whether the packages without any comments are used/needed are welcome.

---

To see which packages are still rather big, use

grep '"size":' ${CONDA_PREFIX}/conda-meta/*.json | sort -k3rn | sed 's/.*conda-meta\///g' | column -t

CC: @isuruf @dimpase @saraedum @mkoeppe @slel

Component: build

Author: Tobias Diez

Issue created by migration from https://trac.sagemath.org/ticket/34626

[mkoeppe]

comment:2

Thanks for investigating this. That's quite a dramatic difference. Could you post a list of the packages in the difference with sizes please?

[mkoeppe]

Some of these (as noted in #35627) are only build dependencies (not runtime dependencies) of some Python packages:

• hatch*, jupyter-packaging, cmake, setuptools_scm, vcversioner,
• patch (only needed in the Sage distribution),
• ninja, meson, meson-python (these will be needed though when we do Switch modularized sagelib's build systems from setuptools to meson-python / ninja #34630)

(None of these help explain the multi-gigabyte difference.)

It may be useful to mark these packages somehow because also in the Sage distribution, we don't actually need to install them into our venv; it suffices to build wheels for them.