Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean compilation #928

Merged
merged 148 commits into from Aug 6, 2021
Merged

Clean compilation #928

merged 148 commits into from Aug 6, 2021

Conversation

EmilyBourne
Copy link
Member

@EmilyBourne EmilyBourne commented Jul 27, 2021

Overhaul the compilation method. A class Compiler is now used to handle all compilation steps. A class CompileObj is used to store all information required to compile a given file. Default compile configurations for basic compilers (gcc, gfortran, icc, ifort) are saved in json files in pyccel/compilers these are determined thanks to the sysconfig module. Fixes #541.

After this PR, tests run much faster. The times below show the last 3 tests run on the master and ebourne_compiling branches, followed by the average times
Master branch

Linux (python3.6) MacOs Windows
19m37 16m54 28m34
18m46 21m13 24m28
21m13 19m53 23m54
19m52 19m20 25m39

ebourne_compiling

Linux (python3.6) MacOs Windows
13m13 17m23 15m17
11m59 16m17 15m25
15m30 17m52 18m01
13m34 17m11 16m11

Supported compilers:
The following compilers have been tested:

  • GNU : gcc/gfortran
  • intel : icc/ifort (tested by @EmilyBourne )
  • PGI : pgcc/pgfortran (tested by @saidctb )
  • nvidia : nvc/nvfort (tested by @rakati )

Commit Summary

  • Create CompileObj to hold all information required to compile a given file
  • Create Compiler to hold all information about a given compiler
  • Remove cwrapper.py. The setup_X.py file is no longer required
  • Use Compiler and CompileObj to simplify pipeline.py and python_wrapper.py
  • Allow multiple accelerators
  • Make mpi an accelerator instead of specifying the mpi_compiler
  • Specify the compiler family (e.g. GNU) instead of the compiler name
  • Add a wrapper_flags argument to separate the flags for the file (fortran) from the flags for the wrapper (c)
  • When compiling a program, compile everything as modules then link separately. This procedure is closer to the procedure used for shared libraries
  • Rename pyc_math to pyc_math_c and pyc_math_f90. This allows us to differentiate between the .o files created from the source files which means we don't need to systematically recompile the library
  • Only overwrite stdlib files if they are out of date
  • Only recompile stdlib files if the source files have been updated
  • Add pyccel.compilers.generate_default to generate the default json files
  • Lock file during pickling. Fixes Pickling and unpickling should use flock #917
  • Add a file describing how to use different compilers

@EmilyBourne
Copy link
Member Author

It seems either impossible or inadvisable to generate the json during the installation. See here

@EmilyBourne EmilyBourne added the Ready_to_merge Approved by senior developer. Ready for final approval and merge label Aug 4, 2021
@yguclu yguclu merged commit 9619ac8 into master Aug 6, 2021
@yguclu yguclu deleted the ebourne_compiling branch August 6, 2021 09:47
yguclu pushed a commit that referenced this pull request Sep 7, 2021
- Remove debugging information
- Correct Windows MPI compiler configuration
- Reactivate unit test on Windows by removing "skipif" flag
- Ensure that files are compiled using `mpif90` when the code is run using `mpi4py` (this was unintentionally removed in PR #928)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ready_to_merge Approved by senior developer. Ready for final approval and merge
Projects
None yet
4 participants