-
Notifications
You must be signed in to change notification settings - Fork 15
Add documentation on the "Building MFEM" page for CMake install #291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
efb7ed6
f6b6bb2
86d8805
ee43a1d
8aaa001
4a1eade
c04a3a0
2357fbb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -239,3 +239,107 @@ git clone https://github.com/spack/spack.git | |||||||||||||
| cd spack | ||||||||||||||
| ./bin/spack install -v mfem | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| ## Building MFEM with CMake | ||||||||||||||
| To build a serial version of MFEM with CMake first create a build directory. For example, using a build directory named `build` inside the MFEM source directory: | ||||||||||||||
| ```sh | ||||||||||||||
| mkdir build | ||||||||||||||
| cd build | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| Run the CMake configuration on the MFEM source directory. | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| Run the build command associated with the CMake configuration, specifying the number of parallel build tasks with the `-j` flag (4 tasks in this case). | ||||||||||||||
| ```sh | ||||||||||||||
| cmake --build . -j 4 | ||||||||||||||
| ``` | ||||||||||||||
| ### Parallel build using CMake | ||||||||||||||
| To build a parallel version of MFEM with CMake first build METIS and Hypre as described above. | ||||||||||||||
| From the MFEM source directory, create a build directory. For example, using a build directory named `build` inside the MFEM source directory: | ||||||||||||||
| ```sh | ||||||||||||||
| mkdir build | ||||||||||||||
| cd build | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| Run the CMake configuration on the MFEM source directory using the `MFEM_USE_MPI` CMake variable to enable MPI. | ||||||||||||||
| This will automatically search for the system MPI implementation, the METIS installation (in `<mfem-source-dir>/../metis-4.0`), and Hypre installation (in `<mfem-source-dir/../hypre`). | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DMFEM_USE_MPI=YES | ||||||||||||||
| ``` | ||||||||||||||
| Alternatively, run the CMake configuration also using the `MFEM_FETCH_TPLS` CMake variable to enable fetching of Hypre and METIS. | ||||||||||||||
| This will automatically download, configure, and build Hypre and METIS alongside MFEM (note that this option is **not** currently supported for GPU builds). | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DMFEM_USE_MPI=YES -DMFEM_FETCH_TPLS=YES | ||||||||||||||
| ``` | ||||||||||||||
| For either CMake configuration approach, now run the build command associated with the configuration, specifying the number of parallel build tasks with the `-j` flag (4 tasks in this case). | ||||||||||||||
| ```sh | ||||||||||||||
| cmake --build . -j 4 | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| ### Advanced configuration steps | ||||||||||||||
| To build with CUDA: | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DMFEM_USE_CUDA=YES | ||||||||||||||
| ``` | ||||||||||||||
| To specify what CUDA architecture to target: | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DCUDA_ARCH="sm_70" | ||||||||||||||
| ``` | ||||||||||||||
| The CUDA architecture is formatted as `sm_{CC}`, or just `{CC}`, where CC is the GPU compute capability of the target GPU without the decimal point. A list of NVIDIA GPU compute capabilities can be found in [the NVIDIA developers documentation](https://developer.nvidia.com/cuda-gpus). Multiple CUDA architectures can be targeted with a comma or semicolon separated list. | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DCUDA_ARCH="{ARCH1},{ARCH2},{ARCH3}" | ||||||||||||||
| ``` | ||||||||||||||
| or | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DCUDA_ARCH="{ARCH1};{ARCH2};{ARCH3}" | ||||||||||||||
| ``` | ||||||||||||||
| Other accepted architecture identifies are `"all"` which targets all CUDA architectures, | ||||||||||||||
| `"all-major"` which targets all major versions `sm_{*0}`, and `"native"` which targets the visible GPUs on the system. | ||||||||||||||
|
|
||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
TODO: currently I think MFEM's CMake setup doesn't allow targeting multiple CUDA architectures at once. I think we should fix this?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is a CMake 3.18 target property Right now it looks like mfem manually sets the The
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I expanded support for Supported formats now include: This should work even for CMake older than 3.18 (the current CMakeLists minimum version is 3.8).
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. looks great, added some documentation for this! |
||||||||||||||
| To build with METIS 5, after following the instructions to build METIS 5 above: | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DMFEM_USE_MPI=YES -DMFEM_USE_METIS_5=YES -DMETIS_DIR=../../metis-5.1.0 | ||||||||||||||
| ``` | ||||||||||||||
| To build with HIP: | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DMFEM_USE_HIP=YES | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| To specify what HIP architecture(s) to target: | ||||||||||||||
| ```sh | ||||||||||||||
| cmake .. -DCMAKE_HIP_ARCHITECTURES="gfx942;gfx90a" | ||||||||||||||
| ``` | ||||||||||||||
| Multiple architectures can be targeted using a semi-colon separated list. | ||||||||||||||
| The HIP architecture for different GPU models can be found in [the LLVM documentation](https://llvm.org/docs/AMDGPUUsage.html#processors) | ||||||||||||||
|
|
||||||||||||||
| When building for GPUs, it is recommended to enable Umpire with `-DMFEM_USE_UMPIRE=ON`; if it is not automatically found by CMake the installation directory can be specified with `-DUMPIRE_DIR=<path-to-umpire-installation-dir>`. Umpire can be downloaded from | ||||||||||||||
|
|
||||||||||||||
gabsillis marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||
| - [https://github.com/LLNL/Umpire/tags](https://github.com/LLNL/Umpire/tags) | ||||||||||||||
|
|
||||||||||||||
| ### Advanced build steps | ||||||||||||||
| Different targets can be built with the --target flag in the build step | ||||||||||||||
| ```sh | ||||||||||||||
| cmake --build . -j 4 --target <target-name> | ||||||||||||||
| ``` | ||||||||||||||
| To build the examples use the `examples` target (the executables will be in the `build/examples` directory). | ||||||||||||||
| ```sh | ||||||||||||||
| cmake --build . -j 4 --target examples | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| To quickly check if the code is successfully built using example 1/1p use the `check` target. | ||||||||||||||
| ```sh | ||||||||||||||
| cmake --build . -j 4 --target check | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| To build the miniapps use the `miniapps` target | ||||||||||||||
| ```sh | ||||||||||||||
| cmake --build . -j 4 --target miniapps | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| To build everything use the `exec` target | ||||||||||||||
| ```sh | ||||||||||||||
| cmake --build . -j 4 --target exec | ||||||||||||||
| ``` | ||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the disclaimer that I'm a bit biased here as the main author of the CMake TPL fetching feature, I'll suggest the option to have only the
-DMFEM_FETCH_TPLS=YESapproach in this documentation to avoid needing to talk about where CMake looks for Hypre and METIS (which obviously includes more than the relative paths used by the pure Makefile system)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Noting that GPU builds are not yet supported with TPL fetching, my suggestion above is moot (i.e., I think the documentation should note both approaches to TPLs)