New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU result has number of walkers dependency and doesn't match CPU result #1054

Open
yaoyi92 opened this Issue Sep 8, 2018 · 4 comments

Comments

Projects
None yet
4 participants
@yaoyi92
Copy link

yaoyi92 commented Sep 8, 2018

Details in the post here.
https://groups.google.com/forum/#!topic/qmcpack/eOE1eIXAgaE

Not 100% sure whether it is a problem for my build. ctest results here in the post
https://groups.google.com/forum/#!topic/qmcpack/1QzNQoceNHs

image

@jtkrogel jtkrogel added the bug label Sep 10, 2018

@yaoyi92

This comment has been minimized.

Copy link

yaoyi92 commented Sep 11, 2018

This seems to be a Volta V100 specific issue. I don't have this problem compiling and running the same simulation on GTX1080.

Need help from someone with experience of Volta V100.

@jtkrogel

This comment has been minimized.

Copy link
Contributor

jtkrogel commented Sep 11, 2018

Something does seem to be up with Volta:
https://cdash.qmcpack.org/CDash/index.php?project=QMCPACK&date=2018-09-07.

Compare "Volta-GCC-CUDA-Release" with "GCC-CUDA-Release".

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Sep 11, 2018

GCC-CUDA-Release runs on a Kepler. This is printed on one on the ~third line of output.

Updated:

Something happened between 7-8 August
Working: https://cdash.qmcpack.org/CDash/testDetails.php?test=2851998&build=26579
Broken: https://cdash.qmcpack.org/CDash/testDetails.php?test=2858752&build=26628
Possibly this was an update to the script or CUDA version, will investigate

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Oct 9, 2018

To keep this updated: the current belief is that this has hit some kind of edge case (bug) or is a newly surfaced problem on Volta or with recent CUDAs (bug). For whatever reason, our current tests don't show the same problem. @PDoakORNL will try to chase down.

@prckent prckent added the gpu label Nov 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment