Skip to content

Embed dependencies in setup.py - #2017

Merged
mrmundt merged 40 commits into
Pyomo:mainfrom
jsiirola:setup-updates
Jun 18, 2021
Merged

Embed dependencies in setup.py#2017
mrmundt merged 40 commits into
Pyomo:mainfrom
jsiirola:setup-updates

Conversation

@jsiirola

@jsiirola jsiirola commented Jun 17, 2021

Copy link
Copy Markdown
Member

Fixes # N/A

Summary/Motivation:

Pyomo currently has dependencies scattered across the GHA / Jenkins builders, .readthedocs-requirements.txt, and setup.py. This PR moves all dependency specification into setup.py, and updates the GHA and RTD builders to pull dependencies from setup.py. As part of the rework, setup.py now specifies three "extras" targets for managing optional dependencies:

  • tests for dependencies needed to run the Pyomo unit tests (currently, nose, coverage, parameterized, and pybind11)
  • docs for dependencies needed to build the RTD documentation
  • optional for packages (available on PyPI) on which Pyomo has an optional dependency.

With this, a reasonable approximation of the old pyomo-extras package is:

pip install 'pyomo[tests,docs,optional]`

The challenge with this PR is that conda does not play well with pip requirements. The complexity here is to provide a bridge whereby our conda builds can interrogate setup.py to get the list of packages that pip would have otherwise installed (and then conda can install from conda-forge).

Changes proposed in this PR:

  • add extras_requires to setup.py
  • add a new command python setup.py dependencies to get lists of required and optional dependency packages
  • rework the GHA builders to get dependencies from setup.py
  • rework RTD configuration to use a .readthedocs.yaml configuration and get documentation requirements from pyomo[docs]

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@jsiirola
jsiirola requested a review from mrmundt June 17, 2021 21:44
fi
# Note: pandas 1.0.3 causes gams 29.1.0 import to fail in python 3.8
PYTHON_PACKAGES="${PYTHON_REQUIRED_PKGS}"
EXTRAS=tests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative way to do this: in the env section, set EXTRAS=test,docs,optional and then for slim, change that. I have no strong opinions, but it's fewer lines.

Comment thread .github/workflows/test_branches.yml Outdated
Comment thread .github/workflows/test_pr_and_main.yml Outdated
Comment thread setup.py Outdated
Comment thread setup.py Outdated
Comment thread setup.py Outdated
@mrmundt
mrmundt self-requested a review June 17, 2021 22:53
@codecov

codecov Bot commented Jun 18, 2021

Copy link
Copy Markdown

Codecov Report

Merging #2017 (6bb2dd3) into main (baf79e5) will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2017      +/-   ##
==========================================
- Coverage   82.70%   82.69%   -0.01%     
==========================================
  Files         592      592              
  Lines       73352    73352              
==========================================
- Hits        60663    60661       -2     
- Misses      12689    12691       +2     
Impacted Files Coverage Δ
pyomo/common/dependencies.py 98.75% <ø> (ø)
pyomo/solvers/plugins/solvers/ASL.py 89.70% <0.00%> (-1.48%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update baf79e5...6bb2dd3. Read the comment docs.

@mrmundt
mrmundt merged commit 4cdb50a into Pyomo:main Jun 18, 2021
@jsiirola
jsiirola deleted the setup-updates branch June 18, 2021 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants