Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use reproducibility flags #14

Merged
merged 4 commits into from
Jan 15, 2020
Merged

Conversation

penguian
Copy link
Contributor

Use reproducibility flags to allow Broadwell (Raijin) and Cascade Lake (Gadi) to give bit-identical results. Tested on Gadi.

Closes #13

@penguian penguian changed the title Document reproducibility flags Use reproducibility flags Dec 20, 2019
NCI_DEBUG_FLAGS = -g3 -O0 -debug all -check all
F90FLAGS_1 = $(NCI_INTEL_FLAGS) $(NCI_REPRO_FLAGS) $(NCI_DEBUG_FLAGS) -CB -no-vec -assume nobuffered_io
CPPDEF = -Duse_netCDF -Duse_comm_$(CHAN) -DTREAT_OVERLAY -DDEBUG -D__VERBOSE
MCT_FCFLAGS = $(NCI_REPRO_FLAGS) $(NCI_DEBUG_FLAGS) -ip -fpe0 -xHost
Copy link
Contributor

@aekiss aekiss Jan 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this use -axCORE-AVX2 instead of -xHost so we can run on Broadwell?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing -xHost implies -axCORE-AVX512 if compiled on the Cascade Lake head nodes, which wouldn't run on Broadwell since that only supports -axCORE-AVX2...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason that I had left the flag on that line as -xHost is that I only tested with DEBUG off. I will need to re-test with DEBUG=yes. Also, if DEBUG=yes, then the flags include -no-vec. I will also double check what is done for the other flag files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will also need to check where FFLAGS and MCT_FCFLAGS are used, since the oasis3-mct apparently uses both.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed by commit a8e4f9b

module load intel-compiler/2019.5.281
module load netcdf/4.7.1
module load openmpi/4.0.1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these changes to config.nci are unnecessary (but harmless), as config.gadi is used in the gadi-transition branch

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed by commit 861be6d

@aekiss
Copy link
Contributor

aekiss commented Jan 15, 2020

Merge 504aaff uses -axCORE-AVX2 from https://github.com/COSIMA/oasis3-mct/tree/use-repro-flags-for-gadi
rather than -xCORE-AVX2 -axSKYLAKE-AVX512,CASCADELAKE from commit f8ee368 to https://github.com/COSIMA/oasis3-mct/tree/gadi-transition

Discussion for a related merge: COSIMA/libaccessom2@a2e2ef2#r36739504

General discussion: COSIMA/access-om2#184

@aekiss aekiss merged commit db46453 into gadi-transition Jan 15, 2020
@access-hive-bot
Copy link

This pull request has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/audit-the-compiler-flags-used-to-build-access-om2-on-gadi-with-the-intel-compiler/437/2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants