New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add GPU AMIP scaling runs #673
Open
juliasloan25
wants to merge
28
commits into
main
Choose a base branch
from
js/gpu-scaling
base: main
Could not load branches
Branch not found: {{ refName }}
Could not load tags
Nothing to show
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
juliasloan25
force-pushed
the
js/gpu-scaling
branch
from
March 6, 2024 18:32
f7a9b60
to
8aeaeed
Compare
juliasloan25
force-pushed
the
js/gpu-scaling
branch
from
March 6, 2024 18:51
8aeaeed
to
41dff6f
Compare
juliasloan25
force-pushed
the
js/gpu-scaling
branch
from
March 7, 2024 01:03
4d1f1a2
to
9d69913
Compare
See here: - SLURM_GPU_BIND: none # https://github.com/open-mpi/ompi/issues/11949#issuecomment-1737712291
+ SLRUM_GRES_FLAGS: "allow-task-sharing" |
juliasloan25
force-pushed
the
js/gpu-scaling
branch
from
March 8, 2024 23:01
48844ec
to
503612c
Compare
juliasloan25
force-pushed
the
js/gpu-scaling
branch
2 times, most recently
from
March 8, 2024 23:39
e8da583
to
c357da3
Compare
juliasloan25
force-pushed
the
js/gpu-scaling
branch
from
March 9, 2024 00:40
c357da3
to
9031045
Compare
juliasloan25
force-pushed
the
js/gpu-scaling
branch
from
March 11, 2024 04:16
2e39025
to
187e958
Compare
juliasloan25
force-pushed
the
js/gpu-scaling
branch
from
March 11, 2024 04:20
187e958
to
f77dfab
Compare
@@ -507,6 +507,9 @@ function update_surface_fractions!(cs::CoupledSimulation) | |||
cs.surface_fractions.ice .= max.(min.(ice_d, FT(1) .- land_s), FT(0)) | |||
cs.surface_fractions.ocean .= max.(FT(1) .- (cs.surface_fractions.ice .+ land_s), FT(0)) | |||
|
|||
comms_ctx = axes(land_s).grid.topology.context |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggested change
comms_ctx = axes(land_s).grid.topology.context | |
comms_ctx = ClimaComms.context(land_s) |
I think we have a helper for this, so that you don't need to use internals.
key: "gpu_amip_dyamond_ws" | ||
command: | ||
- > | ||
julia --threads=3 --color=yes --project=experiments/AMIP experiments/AMIP/coupler_driver.jl |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add an nsys profile
to the non-MPI jobs?
14 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
closes #663
note: we want to use the
slurm_exclusive:
flag to get a GPU that's being used for only the performance job, but this seems not to be working when used in AtmosNeed to use ClimaAtmos#fdf1df4 commit to include PR containing atmos config files. Can't use main because of dependency incompatibilities.
Content