Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update packaging and installation #483

Merged
merged 7 commits into from
Sep 20, 2022
Merged

Update packaging and installation #483

merged 7 commits into from
Sep 20, 2022

Conversation

lfarv
Copy link
Contributor

@lfarv lfarv commented Sep 9, 2022

This updated the packaging and installation setup with two main changes:

Upgrade setuptools:

The main motivation is to ease the compilation of AT with MPI. To allow the modification of the build requirements depending on the context, setuptools now provides in-tree backends: _custom_build/backend.py here. The command for compilation with MPI is now:

MPI=1 pip install ".[mpi]".  # from the local clone
MPI=1 pip install --no-binary accelerator-toolbox "accelerator-toolbox[mpi]"  # from the PyPI repository

Note both the [mpi] option to add mpi4py to the installation requirements, and the MPI=1 environment variable which will add mpi4py to the build requirements and select the mpicc compiler.

As a side effect, most of the configuration moves from setup.cfg to pyproject.toml. Completely removing setup.cfg works but is still considered as beta. So for the time being, we keep a minimal setup.cfg.

Also, setuptools >= 62 cannot be installed on python 3.6. Removing support for python 3.6 also removes some complexity in at/__init__.py.

Move to project to the root of the repository:

  1. This is a common practice to have the project definition at the top-level
  2. This simplifies a lot the installation process: the source code of integrators (atintegrators directory) is now part of the project, so we do not need any more to make copy into pyat/integrator-src. setup.py is simpler and only deals with the definition of extensions.
  3. The idea was also to have a single machine_data package for both Matlab and python by having the top-level machine_data also part of the project. This still does not work because of restrictions in the way setuptools looks for packages. It looks impossible to have package search at two levels, top-level for ./machine_data and pyat/at for the at package. So we still have 2 machine_data repositories.

The difference for users is that pip install must now be run from the top-level instead of from the pyat directory. The documentation is updated.

@lfarv lfarv added the Python For python AT code label Sep 9, 2022
@lfarv
Copy link
Contributor Author

lfarv commented Sep 12, 2022

In addition to the syntax for compiling with MPI involving an environment variable, there is now another syntax:

pip install --config-settings mpi=1 ".[mpi]".  # from the local clone
pip install --no-binary accelerator-toolbox --config-settings mpi=1 "accelerator-toolbox[mpi]"   # from the PyPI repository

This uses recent features of PEP 517. However it's only cosmetic, since the environment variable is still used internally to communicate with setup.py.

This is also valid for compilation with OpenMP:

pip install --config-settings openmp=1  .  # from the local clone
pip install --no-binary accelerator-toolbox --config-settings openmp=1 accelerator-toolbox   # from the PyPI repository

@lfarv
Copy link
Contributor Author

lfarv commented Sep 12, 2022

Rebased on master,

@swhite2401
Copy link
Contributor

Very nice, could you improve the doc page with these informations? I could not find any modifications related to MPI

@lfarv
Copy link
Contributor Author

lfarv commented Sep 14, 2022

@swhite2401 : right, I'll do that.

Copy link
Contributor

@swhite2401 swhite2401 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok after help is updated

@lfarv
Copy link
Contributor Author

lfarv commented Sep 18, 2022

@swhite2401: I'd like to have your feedback on the updated documentation that you can preview here.

@swhite2401
Copy link
Contributor

@swhite2401: I'd like to have your feedback on the updated documentation that you can preview here.

This looks fine to me, at some point we will provide examples but for now we are still working on the multibunch/beamloading devs

@lfarv lfarv merged commit 0502f75 into master Sep 20, 2022
@lfarv lfarv deleted the update_installation branch September 20, 2022 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Python For python AT code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants