Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set up CTest #5068

Merged
merged 183 commits into from
Sep 5, 2024
Merged

Set up CTest #5068

merged 183 commits into from
Sep 5, 2024

Conversation

EZoni
Copy link
Member

@EZoni EZoni commented Jul 20, 2024 •
edited
Loading

Work-in-progress PR to set up CTest. Apply the same logic as in ImpactX and adjust/advance for WarpX.

  • Example of current input files organization:

    • inputs_base_2d
      (base input file for shared input parameters named as inputs_base_<1d;2d;3d;rz>)
    • inputs_test_2d_langmuir_multi_nodal,
      inputs_test_2d_langmuir_multi_psatd,
      ...
      (test input files for test-specific input parameters named as inputs_test_<1d;2d;3d;rz>_..., read base input file through the FILE parameter)
    • test names must start with test_<1d;2d;3d;rz>_... to guarantee consistency and make maintenance easier

  • New Azure CI matrix elements (list coming soon).

  • Run all MPI tests with 2 MPI processes and 1 OpenMP thread. Run single-process tests without MPI.

To-do:

  • WarpX_TEST_CLEANUP to remove test directories
  • WarpX_TEST_FPETRAP for runtime fpe-trapping
  • Resolve FIXME comments or redirect to new issues
  • Avoid copying inputs files ParmParse: Prefix to FILE AMReX-Codes/amrex#4126
  • Enforce new naming convention through CI style check
  • Remove obsolete CI style check scripts
  • Remove obsolete regression testing scripts: CTest follow-up: remove obsolete files/scripts #5220
  • Update docs
  • Remove files after each test: new build flag WarpX_CI
  • Make sure we don't miss any build variant:
  • Add missing checksums (new issue CTest follow-up: add missing checksum files #5206):
    • dirichlet_bc
    • dirichlet_bc_picmi
    • field_probe
    • embedded_boundary_picmi
    • particle_absorption
    • particle_reflection_picmi
    • id_cpu_read_picmi
    • runtime_components_picmi
    • laser_acceleration_rz_opmd
  • Support restart tests
    • Separate input file based on original input file (read through the FILE parameter)
    • Call default regression analysis against original checksum file, with small tolerance
    • Support restart from PICMI (can we set FILE through PICMI?)
    • Update analysis_default_restart.py
  • Support custom run commands (e.g., custom PICMI scripts using lists vs strings)
  • Add stricter dependency between .analysis test and .run test:
    run .analysis test only if .run test passed (now runs if .run test ran)
  • Update test README files (e.g., Ohm solver tests)
  • Skip pyAMReX tests directly from CMakeLists.txt

@EZoni EZoni added enhancement New feature or request component: tests Tests and CI labels Jul 20, 2024
@ax3l ax3l self-requested a review July 22, 2024 17:23
@ax3l ax3l self-assigned this Jul 22, 2024
@EZoni EZoni force-pushed the EZoni_ctest branch 4 times, most recently from 15146f7 to a382488 Compare July 22, 2024 23:25
ax3l
ax3l reviewed Jul 23, 2024
View reviewed changes
Examples/CMakeLists.txt Outdated Show resolved Hide resolved
ax3l
ax3l reviewed Jul 23, 2024
View reviewed changes
Examples/CMakeLists.txt Outdated Show resolved Hide resolved
EZoni
EZoni commented Jul 24, 2024
View reviewed changes
Regression/WarpX-tests.ini Outdated Show resolved Hide resolved
@EZoni EZoni force-pushed the EZoni_ctest branch 4 times, most recently from 09ca93c to a81a95b Compare July 26, 2024 00:23
This was referenced Sep 3, 2024
Open
Closed
@ax3l ax3l mentioned this pull request Sep 3, 2024
Merged
5 tasks
ax3l
ax3l reviewed Sep 3, 2024
View reviewed changes
Examples/CMakeLists.txt Outdated Show resolved Hide resolved
ax3l
ax3l reviewed Sep 4, 2024
View reviewed changes
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
Docs/source/developers/testing.rst Outdated Show resolved Hide resolved
@EZoni EZoni mentioned this pull request Sep 5, 2024
Open
WeiqunZhang pushed a commit to AMReX-Codes/amrex that referenced this pull request Sep 5, 2024
@ax3l
ParmParse: Prefix to FILE (#4126)
216ce6f 
## Summary

For CI/CD workflows and out-of-source tests we often want to include
dependent inputs files via `FILE = <filename>`. For development, we
prefer to run in temporary run directories but want to avoid having to
copy over the latest inputs file from a source directory (mostly to
avoid confusion between source and copy and to enable rapid development
cycles).

Now, the environment variable `AMREX_INPUTS_FILE_PREFIX` can be set to
prefix every `FILE = <filename>` with a custom path. We will use this in
the CTests integration of WarpX.

## Additional background

CC @EZoni
ECP-WarpX/WarpX#5068
ax3l
ax3l reviewed Sep 5, 2024
View reviewed changes
CMakeLists.txt Show resolved Hide resolved
ax3l
ax3l approved these changes Sep 5, 2024
View reviewed changes
Copy link
Member

@ax3l ax3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, fantastic work! 🚀 ✨

@ax3l ax3l enabled auto-merge (squash) September 5, 2024 22:57
@ax3l ax3l merged commit e4cd1aa into ECP-WarpX:development Sep 5, 2024
38 checks passed
@EZoni EZoni mentioned this pull request Sep 6, 2024
Merged
@EZoni EZoni deleted the EZoni_ctest branch September 6, 2024 22:07
@EZoni EZoni mentioned this pull request Sep 10, 2024
Merged
ax3l added a commit that referenced this pull request Sep 13, 2024
@ax3l
CTest: No Backtrace Handler
e601a57 
Missing in #5068.

See:
- https://github.com/ECP-WarpX/impactx/blob/71d0e5b3f7a5e27f337c303c1fde61700f5ea702/examples/CMakeLists.txt#L82-L83
- https://amrex-codes.github.io/amrex/docs_html/Debugging.html#breaking-into-debuggers
@ax3l ax3l mentioned this pull request Sep 13, 2024
Open
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ax3l ax3l ax3l approved these changes

Assignees

@ax3l ax3l

Labels
component: tests Tests and CI enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@EZoni @ax3l