Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single build for Python 2 and 3 #256

Closed
wants to merge 4 commits into from
Closed

Single build for Python 2 and 3 #256

wants to merge 4 commits into from

Conversation

evertrol
Copy link

@evertrol evertrol commented Jun 26, 2018

(do not merge; preview)

This is a preview for a PR for a single build system that works for both Python 2 and Python 3.
This removes the support3/ directory, and makes the modules in support/ Python 3, with Python 2 backward compatibility.
Build files such as setup.py and Makefile also remove their dependencies on support3.

At the moment, many parts are not built (i.e., less parts are built than with the support3 package), for both Python 2 and 3. There'll be parts I've overlooked in the conversion, but I've put the PR here so people can have a peek.

At the moment, during a build, the 2to3 script is run to convert all the Amuse code to Python 3 (so this can be built in-place). This change can afterwards be easily undone with git checkout -- src.

@ipelupessy
Copy link
Member

I think in case of python 3 you don't want in place build...only build and install

Evert Rol added 4 commits June 27, 2018 14:47
This makes the files under support/ mostly Python 3, with backward compatibility for Python 2.

Changes include

- `from __future__ import print_function` and `print(...)`

- imports with `ModuleNotFoundError` or `ImportError` catches(*)

- `.iteritems()` and `.itervalues()` replacead by `items()` and
  `values()` (this assumes there are no large dicts involved).

- `map` replaced by a list comprehension.

- a few specific changes, such as the `func_code` vs. the `__code__` attribute.

(*) Which exception is tested for, depends on what is the easiest to
test for. `ModuleNotFoundError` should be caught for if the first
attempted imports are for Python 2, `ImportError` should be caught for
if the attempted imports are for Python 3 instead.
Build parts for 'framework' and a single code are now the same for Python 2 and 3.

This does rely more on a hack than proper Python 3 support: because
the Python 2 Amuse code code needs to be compiled to Python 3, but not
in place (to not mess up the (git) source), the necessary Python files
are copied to the temporary build directory, and then compiled. This
means that, to actually use the code, one should set their PYTHONPATH
to the relevant build directory (this happens automatically when
building though, i.e., when using `make <something>.code`).

Compared to building all codes at once
(support/setup_codes.py/BuildCodes or BuildLibraries), which uses
copy_codes_to_build_dir, this copies once, but avoids copying again if
a file is already found in the build dir (and presumed compiled for
the correct Python version).

This work-around can be avoided in the future in two ways:

- the core Amuse code is Python 3, and backwards compatible with
  Python 2. In that case, no copying of files is necessary.

- the code Amuse code is its own package, and becomes an essential
  requirement to install before using the generic Amuse package. In
  this case, the installation of the core Amuse code will have ensure
  the code is already compiled for the relevant Python version.
@evertrol
Copy link
Author

This now provides single build mechanisms for Python 2 and 3. For example, run configure with the relevant Python executable set:

PYTHON=/usr/local/bin/python3.6 ./configure

and each of the following works:

make build
make framework
make <something>.code

At the moment, all Python 2 builds are in-place (as was the default), while all Python 3 builds are out-of-place (to not mess up the git source tree). See also #271 .

The work-around for building a single code is similar as for setup.py build_codes, since the relevant Python files have to be copied first to a build dir, before converted to Python 3. I've tried to copy only the relevant files, and avoid copying and converting them again if another single code is build. But "relevant files" may be too few, so perhaps all files (*.py and a few *.template files) should be recursively copied (the codes/ subdirectory could be excluded, but not so e.g., the community/ subdir).
See also the commit message for fc6682d

@evertrol
Copy link
Author

evertrol commented Jun 27, 2018

Might be good to move the last commit (fc6682d) into a separate PR. That new PR will then address #186 more explicitly.

@evertrol
Copy link
Author

Replaced by #273 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants