Dirac converter with MSD wave functions #3510

camelto2 · 2021-10-04T20:53:21Z

Please review the developer documentation
on the wiki of this project that contains help and requirements.

Proposed changes

This PR expands convert4qmc for DIRAC to store CI wave functions coming from complete open-shell CI (COSCI) commonly used in DIRAC. The xml files are also modified to accommodate MSD for spinors wavefunctions. This makes the DIRAC conversion significantly more user friendly since the user does not have to specify the CI expansion by hand.

Note: Typically in COSCI, many different CI expansions are present in output file. The wavefunction that is stored in the hdf5 can be controlled by the -TargetState x option in convert4qmc, otherwise it defaults to the first state that is found. At the moment, only one state is stored in the hdf5. This is because the QMCGaussianParserBase base class currently only supports one CI state if we are parsing the output file to generate the hdf5 file, despite the fact that QMCPACK can handle many CI stored in the hdf5 and select it with the ext_level flag as is done with Quantum Package. A later PR can update QMCGaussianParserBase to hold arbitrarily many CI states...this would allow gamess and dirac conversions to store all the states in the output files to the hdf5. However for the time being, having only one state is sufficient.

Note 2: In order to accommodate the MSD storage in the hdf5 and updates to the xml file, I had to update the converter test "gold" files. This is turn required me to update the multi_slater_determinant unit tests that used the "gold.orbs.h5" from the converter test. I modified things so that the converter test checks if I can store and arbitrary excited state (state 14 in this case) and the multi_slater_determinant unit tests reference values were updated accordingly. A new test is also added to make sure the CI expansion can be read directly from the hdf5 in addition to the xml.

What type(s) of changes does this code introduce?

Delete the items that do not apply

New feature
Testing changes (e.g. new unit/integration/performance tests)
Documentation changes

Does this introduce a breaking change?

No

What systems has this change been tested on?

my workstation

Checklist

Update the following with a yes where the items apply. If you're unsure about any of them, don't hesitate to ask. This is
simply a reminder of what we are going to look for before merging your code.

Yes. This PR is up to date with current the current state of 'develop'
Yes. Code added or changed in the PR has been clang-formatted
Yes. This PR adds tests to cover any new code, or to catch a bug that is being fixed
Yes. Documentation has been added (if appropriate)

I updated the gold.orbs.h5 to use convert4qmc -dirac df_Bi.out -TargetState 14 -prefix gold -nojastrow This tests the converter to make sure it can grab an arbitrary excited state to store in the hdf5. However, this change made it so that I needed to update the multi_slater unit tests, which relied on this gold.orbs.h5 file. Also, added to the unit test is reading the CI coeffs directly from the h5 file

qmc-robot · 2021-10-04T20:54:02Z

Can one of the admins verify this patch?

prckent · 2021-10-05T19:43:21Z

Memory leaks were caught in the converter:

52/742 Test   #52: converter_test_Bi_dirac ..........................................................................***Failed  Error regular expression found in output. Regex=[  FAIL]  0.73 sec
Return code nonzero:  1
Stderr not empty

=================================================================
==2287==ERROR: LeakSanitizer: detected memory leaks

Indirect leak of 120 byte(s) in 1 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30cfa07cb in xmlNewText (/lib/x86_64-linux-gnu/libxml2.so.2+0x5e7cb)

Indirect leak of 120 byte(s) in 1 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30cfa0564 in xmlNewNode (/lib/x86_64-linux-gnu/libxml2.so.2+0x5e564)

Indirect leak of 96 byte(s) in 1 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30cfa09a1 in xmlNewProp (/lib/x86_64-linux-gnu/libxml2.so.2+0x5e9a1)

Indirect leak of 25 byte(s) in 3 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30d014529 in xmlStrdup (/lib/x86_64-linux-gnu/libxml2.so.2+0xd2529)

SUMMARY: AddressSanitizer: 361 byte(s) leaked in 6 allocation(s).

ye-luo · 2021-10-05T20:12:39Z

@williamfgc should we add CMAKE_CXX_FLAGS=-g to get source code line number printed when asan failure shows up?

camelto2 · 2021-10-05T20:16:41Z

@williamfgc should we add CMAKE_CXX_FLAGS=-g to get source code line number printed when asan failure shows up?

This would be helpful. I built on my mac with -DENABLE_SANITIZER=asan and I am not seeing the same failures. Having the line numbers shown would help me find the issue much faster

camelto2 · 2021-10-05T23:29:52Z

Memory leaks were caught in the converter:

52/742 Test   #52: converter_test_Bi_dirac ..........................................................................***Failed  Error regular expression found in output. Regex=[  FAIL]  0.73 sec
Return code nonzero:  1
Stderr not empty

=================================================================
==2287==ERROR: LeakSanitizer: detected memory leaks

Indirect leak of 120 byte(s) in 1 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30cfa07cb in xmlNewText (/lib/x86_64-linux-gnu/libxml2.so.2+0x5e7cb)

Indirect leak of 120 byte(s) in 1 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30cfa0564 in xmlNewNode (/lib/x86_64-linux-gnu/libxml2.so.2+0x5e564)

Indirect leak of 96 byte(s) in 1 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30cfa09a1 in xmlNewProp (/lib/x86_64-linux-gnu/libxml2.so.2+0x5e9a1)

Indirect leak of 25 byte(s) in 3 object(s) allocated from:
    #0 0x49c05d in malloc (/__w/qmcpack/qmcpack-build/bin/convert4qmc+0x49c05d)
    #1 0x7fb30d014529 in xmlStrdup (/lib/x86_64-linux-gnu/libxml2.so.2+0xd2529)

SUMMARY: AddressSanitizer: 361 byte(s) leaked in 6 allocation(s).

should be fixed now

prckent · 2021-10-05T23:34:31Z

Thanks Cody. We need to find a more convenient development route...

ye-luo · 2021-10-05T23:36:21Z

Test this please

camelto2 · 2021-10-05T23:36:23Z

Thanks Cody. We need to find a more convenient development route...

this would have been easier to find had I compiled libxml2 with -g as opposed to using what was already on the system

commit 22cc717 Author: Peter Doak <doakpw@ornl.gov> Date: Wed Oct 6 13:05:22 2021 -0400 propagating mw_invertPsiM const API changes commit fea01c4 Merge: 4e8b3f4 f7874e4 Author: Peter Doak <doakpw@ornl.gov> Date: Wed Oct 6 12:38:07 2021 -0400 Merge branch 'develop' into matrix_update_engines_direct_2 commit f7874e4 Merge: 319938f 3a1fc75 Author: Ye Luo <yeluo@anl.gov> Date: Wed Oct 6 10:57:22 2021 -0500 Merge pull request QMCPACK#3514 from ye-luo/fix-omp-nocuda Fix omp offload without cuda commit 3a1fc75 Merge: 4b2ef72 319938f Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 21:49:12 2021 -0500 Merge branch 'develop' into fix-omp-nocuda commit 319938f Merge: c093820 1a71e25 Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 21:27:24 2021 -0500 Merge pull request QMCPACK#3510 from camelto2/dirac_converter_msd Dirac converter with MSD wave functions commit 4b2ef72 Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 20:43:26 2021 -0500 Mark all the RefVector const. commit 3191d09 Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 20:38:22 2021 -0500 Revert integration with DiracMatrixComputeOMPTarget commit 1a71e25 Merge: 9414af2 c093820 Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 18:36:14 2021 -0500 Merge branch 'develop' into dirac_converter_msd commit 9414af2 Author: camelto2 <cmelton@sandia.gov> Date: Tue Oct 5 17:28:37 2021 -0600 fix libxml leak commit c093820 Merge: 132b560 309afa1 Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 16:55:34 2021 -0500 Merge pull request QMCPACK#3513 from quantumsteve/available_supported cif2cell_supported not set commit 309afa1 Merge: 94b98f5 132b560 Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 15:51:21 2021 -0500 Merge branch 'develop' into available_supported commit 132b560 Merge: 67490d9 486884f Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 15:28:08 2021 -0500 Merge pull request QMCPACK#3512 from ye-luo/fix-StlPrettyPrint Collapsed style in StlPrettyPrint. commit 94b98f5 Author: Steven Hahn <hahnse@ornl.gov> Date: Tue Oct 5 15:02:00 2021 -0400 copy/paste error Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit 486884f Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 13:47:02 2021 -0500 Remove unused header inclusion. commit a468c4a Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 01:12:10 2021 -0500 Collapsed style in StlPrettyPrint. commit 67490d9 Merge: aca1bf6 4ea1d47 Author: Ye Luo <yeluo@anl.gov> Date: Tue Oct 5 12:22:50 2021 -0500 Merge pull request QMCPACK#3511 from quantumsteve/cif2cell_available Check if cif2cell is available before running ntest_nexus_structure commit 4ea1d47 Author: Steven Hahn <hahnse@ornl.gov> Date: Tue Oct 5 11:02:22 2021 -0400 Add Python 3.10.0 Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit 20d2e84 Author: Steven Hahn <hahnse@ornl.gov> Date: Tue Oct 5 10:03:43 2021 -0400 Check if cif2cell is available before running test Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit bbfe24f Author: camelto2 <cmelton@sandia.gov> Date: Mon Oct 4 16:09:36 2021 -0600 fix uninnitialized warning commit ee35c16 Author: camelto2 <cmelton@sandia.gov> Date: Mon Oct 4 14:51:18 2021 -0600 remove unused code in DiracParser commit 0983b0a Author: camelto2 <cmelton@sandia.gov> Date: Mon Oct 4 14:18:26 2021 -0600 update manual entry for convert4qmc with DIRAC commit 8e064a0 Author: camelto2 <cmelton@sandia.gov> Date: Mon Oct 4 14:11:11 2021 -0600 clang formatting commit 806d493 Author: camelto2 <cmelton@sandia.gov> Date: Mon Oct 4 14:04:21 2021 -0600 update unit and converter tests to reflect changes in MSD for dirac I updated the gold.orbs.h5 to use convert4qmc -dirac df_Bi.out -TargetState 14 -prefix gold -nojastrow This tests the converter to make sure it can grab an arbitrary excited state to store in the hdf5. However, this change made it so that I needed to update the multi_slater unit tests, which relied on this gold.orbs.h5 file. Also, added to the unit test is reading the CI coeffs directly from the h5 file commit 3c48afd Merge: a4ebddb aca1bf6 Author: camelto2 <cmelton@sandia.gov> Date: Mon Oct 4 09:30:53 2021 -0600 Merge remote-tracking branch 'upstream/develop' into dirac_converter_msd commit aca1bf6 Merge: 618cdb2 2318723 Author: Paul R. C. Kent <kentpr@ornl.gov> Date: Sun Oct 3 13:02:42 2021 -0400 Merge pull request QMCPACK#3505 from anbenali/Convert4qmcGuessH5Format Convert4qmc guess h5 format commit 2318723 Author: Anouar Benali <abenali.sci@gmail.com> Date: Sun Oct 3 01:09:41 2021 -0500 fixing passing an hdf5 file without the -orbitals tag. closing bug # QMCPACK#3503 commit 273ecb0 Author: Anouar Benali <abenali.sci@gmail.com> Date: Sat Oct 2 23:57:51 2021 -0500 Change default behavior of convert4qmc: Will only generate inputs with Jastrows unless -nojastrow is specified, then only inputs with no jastrow will be generated commit 618cdb2 Merge: c0ed5f1 0a13c01 Author: Paul R. C. Kent <kentpr@ornl.gov> Date: Sat Oct 2 13:32:16 2021 -0400 Merge pull request QMCPACK#3502 from ye-luo/fix-nightly Fix and update nightly test scripts. commit 0a13c01 Author: Ye Luo <yeluo@anl.gov> Date: Sat Oct 2 11:29:44 2021 -0500 Fix and update nightly test scripts. commit c0ed5f1 Merge: 4b9b609 b7ce2c8 Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 18:33:06 2021 -0500 Merge pull request QMCPACK#3501 from ye-luo/fix-hip Fix HIP CMake commit b7ce2c8 Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 17:27:49 2021 -0500 Fix HIP CMake. commit 4b9b609 Merge: 91fa888 4042431 Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 15:37:41 2021 -0500 Merge pull request QMCPACK#3489 from prckent/ppconvertcmakeclean Fix ppconvert memory bugs and enable in testing commit 4042431 Merge: d48bd76 91fa888 Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 14:37:23 2021 -0500 Merge remote-tracking branch 'origin/develop' into ppconvertcmakeclean commit d48bd76 Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 14:37:06 2021 -0500 Update README.md about CUDA_ARCH commit 91fa888 Author: William F Godoy <williamfgc@yahoo.com> Date: Wed Sep 29 11:10:37 2021 -0400 Add docs for macOS CI commit fe5aa2d Author: William F Godoy <williamfgc@yahoo.com> Date: Wed Sep 29 09:46:20 2021 -0400 Add Python dependencies on macOS runner commit 60724b9 Author: William F Godoy <williamfgc@yahoo.com> Date: Tue Sep 28 17:06:13 2021 -0400 Reduce brew dependencies commit 72d4c01 Author: William F Godoy <williamfgc@yahoo.com> Date: Tue Sep 28 16:39:07 2021 -0400 Add macOS CI on GitHub Actions gcc-11 real build brew dependencies commit 7ee1a1d Merge: d008890 a4bb505 Author: Paul R. C. Kent <kentpr@ornl.gov> Date: Fri Oct 1 13:37:48 2021 -0400 Merge pull request QMCPACK#3492 from quantumsteve/cudatoolkit Eliminate deprecated find_package(CUDA) from qmcpack commit a4ebddb Author: camelto2 <cmelton@sandia.gov> Date: Fri Oct 1 10:43:24 2021 -0600 fix for spinors in h5 and xml for MSD commit a4bb505 Merge: d947ca1 d008890 Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 12:08:40 2021 -0400 Merge remote-tracking branch 'origin/develop' into cudatoolkit commit d947ca1 Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 12:08:26 2021 -0400 Update build_olcf_summit_Clang.sh commit d008890 Merge: a556925 d546e4e Author: Ye Luo <yeluo@anl.gov> Date: Fri Oct 1 11:08:04 2021 -0500 Merge pull request QMCPACK#3498 from ye-luo/opt-J2 Fix and optimize offload J2 commit 3884a95 Author: camelto2 <cmelton@sandia.gov> Date: Thu Sep 30 22:27:46 2021 -0600 correct MSD xml generated commit d546e4e Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 22:19:53 2021 -0500 Minimize recompute in J2. commit 517dea7 Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 01:14:49 2021 -0500 Cure non-determinisitic offload J2. Reproducer: NiO a64 batched_driver performance test. Run 1 VMC step with 1 thread over and over. The scalar.dat is not deterministic. Kinetic is different. mw_updateVGL. Inject print before and after the offload region. walker 13 and electron 741. Sometimes the value is not updated even if a walker is accepted. commit 587634d Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 19:10:31 2021 -0500 Update installation.rst commit f8ed8db Merge: fe7fba7 a556925 Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 19:01:19 2021 -0500 Merge remote-tracking branch 'origin/develop' into cudatoolkit commit fe7fba7 Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 19:00:23 2021 -0500 Allow OFFLOAD_ARCH not being set for NVHPC. commit a9e69a1 Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 18:43:55 2021 -0500 Make NVHPC support CMAKE_CUDA_ARCHITECTURES as a list. commit be7893e Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 16:58:44 2021 -0500 Our header only wrappers needs cuda include path. commit 9b36461 Merge: 575ea73 34953ef Author: camelto2 <cmelton@sandia.gov> Date: Thu Sep 30 14:38:47 2021 -0600 Merge remote-tracking branch 'upstream/develop' into dirac_converter_msd commit 575ea73 Author: camelto2 <cmelton@sandia.gov> Date: Thu Sep 30 14:37:43 2021 -0600 enable complete open shell CI parsing for DIRAC converter commit 8f1b5af Author: Steven Hahn <hahnse@ornl.gov> Date: Thu Sep 30 16:30:05 2021 -0400 update documentation Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit afd2294 Author: Steven Hahn <hahnse@ornl.gov> Date: Thu Sep 30 16:06:50 2021 -0400 Check LLVM offload only contains one architecture Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit 3ca4e31 Merge: 3ba75e9 b5571ec Author: Paul R. C. Kent <kentpr@ornl.gov> Date: Thu Sep 30 14:51:29 2021 -0400 Merge pull request #1 from williamfgc/ppconvertcmakeclean-fix-leaks Ppconvertcmakeclean fix memory leaks commit b5571ec Author: William F Godoy <williamfgc@yahoo.com> Date: Thu Sep 30 13:46:49 2021 -0400 Revert clang-format related changes commit 9b2e2fb Author: William F Godoy <williamfgc@yahoo.com> Date: Thu Sep 30 11:47:07 2021 -0400 Fix leaks associated with Grid raw pointer Use std::shared_ptr to enable copy constructors commit 216d9dd Author: William F Godoy <williamfgc@yahoo.com> Date: Thu Sep 30 10:47:27 2021 -0400 Address leaks in XMLWriterClass Use std::shared_ptr to allow for deep copy constructors commit 058e446 Author: Ye Luo <yeluo@anl.gov> Date: Thu Sep 30 10:06:18 2021 -0400 Update recipes under config for CUDA change. commit b39aea6 Author: Ye Luo <yeluo@anl.gov> Date: Wed Sep 29 19:06:57 2021 -0500 Set CMAKE_CUDA_ARCHITECTURES early. commit ebf7849 Author: Ye Luo <yeluo@anl.gov> Date: Wed Sep 29 19:02:47 2021 -0500 Make platform_cuda_legacy depend on CUDA::cudart commit 81457ad Merge: 57e55d3 ed14d86 Author: Ye Luo <yeluo@anl.gov> Date: Wed Sep 29 17:52:01 2021 -0500 Merge remote-tracking branch 'origin/develop' into cudatoolkit commit 57e55d3 Author: Ye Luo <yeluo@anl.gov> Date: Wed Sep 29 17:30:50 2021 -0500 Set back CUDA default to C++14. commit 734415c Author: Ye Luo <yeluo@anl.gov> Date: Wed Sep 29 17:24:08 2021 -0500 CMAKE_CUDA_ARCHITECTURES needs CMake 3.18. commit 707977e Author: Ye Luo <yeluo@anl.gov> Date: Wed Sep 29 17:22:44 2021 -0500 More accurate stopper message. commit 761745b Author: Steven Hahn <hahnse@ornl.gov> Date: Wed Sep 29 17:36:10 2021 -0400 Apply changes recommended by @ye-lou. Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit b607510 Author: Steven Hahn <hahnse@ornl.gov> Date: Wed Sep 29 15:06:19 2021 -0400 Don't change required CMake version Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit ed13bd8 Author: Steven Hahn <hahnse@ornl.gov> Date: Fri Sep 24 10:47:26 2021 -0400 Eliminate depricated find_package(CUDA) from qmcpack Replace it with first-class language support and find_package(CUDAToolkit) Signed-off-by: Steven Hahn <hahnse@ornl.gov> commit 3ba75e9 Author: Paul Kent <kentpr@ornl.gov> Date: Tue Sep 28 22:04:05 2021 -0400 Test label commit dc39dba Author: Paul Kent <kentpr@ornl.gov> Date: Tue Sep 28 21:55:32 2021 -0400 Try public in cmake

camelto2 added 9 commits September 30, 2021 14:37

enable complete open shell CI parsing for DIRAC converter

575ea73

Merge remote-tracking branch 'upstream/develop' into dirac_converter_msd

9b36461

correct MSD xml generated

3884a95

fix for spinors in h5 and xml for MSD

a4ebddb

Merge remote-tracking branch 'upstream/develop' into dirac_converter_msd

3c48afd

clang formatting

8e064a0

update manual entry for convert4qmc with DIRAC

0983b0a

remove unused code in DiracParser

ee35c16

fix uninnitialized warning

bbfe24f

fix libxml leak

9414af2

Merge branch 'develop' into dirac_converter_msd

1a71e25

ye-luo approved these changes Oct 6, 2021

View reviewed changes

ye-luo merged commit 319938f into QMCPACK:develop Oct 6, 2021

camelto2 deleted the dirac_converter_msd branch September 12, 2022 18:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dirac converter with MSD wave functions #3510

Dirac converter with MSD wave functions #3510

camelto2 commented Oct 4, 2021

qmc-robot commented Oct 4, 2021

prckent commented Oct 5, 2021

ye-luo commented Oct 5, 2021

camelto2 commented Oct 5, 2021 •

edited

camelto2 commented Oct 5, 2021

prckent commented Oct 5, 2021

ye-luo commented Oct 5, 2021

camelto2 commented Oct 5, 2021 •

edited

Dirac converter with MSD wave functions #3510

Dirac converter with MSD wave functions #3510

Conversation

camelto2 commented Oct 4, 2021

Proposed changes

What type(s) of changes does this code introduce?

Does this introduce a breaking change?

What systems has this change been tested on?

Checklist

qmc-robot commented Oct 4, 2021

prckent commented Oct 5, 2021

ye-luo commented Oct 5, 2021

camelto2 commented Oct 5, 2021 • edited

camelto2 commented Oct 5, 2021

prckent commented Oct 5, 2021

ye-luo commented Oct 5, 2021

camelto2 commented Oct 5, 2021 • edited

camelto2 commented Oct 5, 2021 •

edited

camelto2 commented Oct 5, 2021 •

edited