Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cmake option to not build OpenMP by default. #11

Merged
merged 2 commits into from
Feb 16, 2021

Conversation

jamiebresch
Copy link

On Cheyenne, when submitting jobs (for example, #PBS -l select=6:ncpus=32:mpiprocs=32), OMP_NUM_THREADS will default to the value of ncpus (=32 in this example). Either add ompthreads=xx setting to PBS, or setenv OMP_NUM_THREADS xx in the script to specify desired threads.
When mpas/mpas_jedi is compiled with OpenMP and the job is run with more than 1 OMP_NUM_THREADS, it hangs somewhere after ----- done checking limited-area settings ----- in log.atmosphere.0000.out.
This PR changes the default behavior of cmake from Enable OpenMP to disabled.

Copy link

@mmiesch mmiesch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks reasonable, though I have not tested it

Copy link
Collaborator

@svahl991 svahl991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on Cheyenne with GNU.

@liujake
Copy link
Collaborator

liujake commented Feb 11, 2021

FYI. I tried to build the model inside mpas-bundle without OpenMP using jedi-gnu modules. This does NOT solve the problem: the executable still can not do 30-km mesh forecast. For my case, it hangs at

 Reading streams configuration from file streams.atmosphere
 Found mesh stream with filename template static.nc
 Using default io_type for mesh stream
  ** Attempting to bootstrap MPAS framework using stream: static
 Bootstrapping framework with mesh fields from input file 'static.nc'

When I build the same source code with the traditional way using intel+mpt, it can run the forecast with the same initial condition file.

Related issues are documented at https://github.com/JCSDA-internal/mpas-bundle/issues/90.

@jjguerrette
Copy link

jjguerrette commented Feb 16, 2021

Is this ready to merge?

@jamiebresch
Copy link
Author

Is this ready to merge?

I think so.

@jjguerrette jjguerrette merged commit 651728e into release-stable Feb 16, 2021
@jjguerrette jjguerrette deleted the feature/disable_openmp_cmake branch February 16, 2021 16:47
@jjguerrette
Copy link

FYI. I tried to build the model inside mpas-bundle without OpenMP using jedi-gnu modules. This does NOT solve the problem: the executable still can not do 30-km mesh forecast. For my case, it hangs at

 Reading streams configuration from file streams.atmosphere
 Found mesh stream with filename template static.nc
 Using default io_type for mesh stream
  ** Attempting to bootstrap MPAS framework using stream: static
 Bootstrapping framework with mesh fields from input file 'static.nc'

When I build the same source code with the traditional way using intel+mpt, it can run the forecast with the same initial condition file.

Related issues are documented at JCSDA-internal/mpas-bundle#90.

@liujake, I started seeing this same hanging issue in some mpas_hofx_nomodel runs. It is unrelated to the MPI_Finalize issue that is documented in JCSDA-internal/mpas-bundle#90. I will try a new build soon to see if the issue goes away.

climbfuji pushed a commit that referenced this pull request Aug 26, 2022
On Cheyenne, when submitting jobs (for example, #PBS -l select=6:ncpus=32:mpiprocs=32), OMP_NUM_THREADS will default to the value of ncpus (=32 in this example). Either add ompthreads=xx setting to PBS, or setenv OMP_NUM_THREADS xx in the script to specify desired threads.
When mpas/mpas_jedi is compiled with OpenMP and the job is run with more than 1 OMP_NUM_THREADS, it hangs somewhere after  `----- done checking limited-area settings -----` in log.atmosphere.0000.out.
This PR changes the default behavior of cmake from Enable OpenMP to disabled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants