Cusp Correction for SOA - Serial #1172

anbenali · 2018-11-26T18:05:43Z

Cleaned print statments from Mark
Store xml file with Coefficients
Generate Coefficients if not found
Serial implementation for now

qmc-robot · 2018-11-26T18:07:01Z

Can one of the maintainers verify this patch?

qmc-robot · 2018-11-26T18:07:07Z

Can one of the maintainers verify this patch?

prckent

Thanks. I know that this is currently a WIP, but clearly doxygen comments are needed for all these functions. Additional comments throughout would be beneficial.

e.g. What are the magic numbers in getOneIdealLocalEnergy and is it significant that they are only there is 6 s.f.?

ye-luo · 2018-11-26T18:15:59Z

src/QMCWaveFunctions/lcao/LCAOrbitalBuilder.cpp

@@ -103,8 +103,8 @@ namespace qmcplusplus
  }


-  LCAOrbitalBuilder::LCAOrbitalBuilder(ParticleSet& els, ParticleSet& ions, Communicate *comm, xmlNodePtr cur)
-    : SPOSetBuilder(comm), targetPtcl(els), sourcePtcl(ions), myBasisSet(nullptr), h5_path(""), doCuspCorrection(false)
+  LCAOrbitalBuilder::LCAOrbitalBuilder(ParticleSet& els, ParticleSet& ions, xmlNodePtr cur)


This is a regression.

src/QMCWaveFunctions/lcao/LCAOrbitalBuilder.cpp

ye-luo · 2018-11-26T18:19:09Z

src/QMCWaveFunctions/lcao/LCAOrbitalBuilder.cpp

@@ -516,32 +670,28 @@ namespace qmcplusplus
    LCAOrbitalSet *lcos = nullptr;
    LCAOrbitalSetWithCorrection *lcwc = nullptr;
    if (doCuspCorrection) {
-      lcwc =new LCAOrbitalSetWithCorrection(sourcePtcl, targetPtcl, myBasisSet, rank()==0);
+      lcwc =new LCAOrbitalSetWithCorrection(sourcePtcl, targetPtcl, myBasisSet, ReportLevel);


This is a regression.

ye-luo · 2018-11-26T18:24:55Z

In the mixed precision case, we probably need to do the cusp correction construction in full precision while do the evaluation in single. Do we have a plan how to do this?

prckent · 2018-11-26T18:34:49Z

Re: mixed-precision. I am also interested in the plan, but for an initial implementation I wouldn't object if we only supported double. A working executable path is needed in SoA to help debug #1145 .

prckent · 2018-11-26T20:10:13Z

OK to test

anbenali · 2018-11-26T20:15:46Z

Keep in Mind that the point is to have this available for now for anyone "stuck with a bug" and also to help find errors if any.

I am making the PR because I needed this code, but all the credit goes to @markdewing who made 99.99% of the work. (which means I cannot make the doc for the code)

markdewing · 2018-11-26T20:46:21Z

I will write some documentation for these functions.

prckent · 2018-11-26T20:49:37Z

Thanks @markdewing
@anbenali : remove the WIP when you are finished updating

anbenali · 2018-11-26T21:31:15Z

@prckent I am temptd to stage this in 2 separate PRs. one for Serial and one for parallel implementation just because of the urgency of the matter. I am testing the code on a very large set of molecules and in some cases there is significant drop in the variance and multidet is slightly stabilized from large fluctuations.
If that is fine with you I rather do it that way. Otherwise it will be one more week I guess before I can touch the parallel part..

prckent · 2018-11-26T21:34:04Z

@anbenali I assumed this was your plan from the PR name. When the serial implementation is in we can already begin to learn about #1145 and other surprises. Running faster comes later.

prckent · 2018-11-28T21:59:55Z

@markdewing Do you plan to update the code in this PR with some comments, or should we merge now and update later?
@anbenali is this ready to merge apart from the docs? Remove the WIP if so.

Also remove some unused code.

markdewing · 2018-11-28T22:18:40Z

I'm working on some comments, documentation, and removing some unused code. I can probably get that into this branch today.

into CuspCorrection_SOA

ye-luo · 2018-11-28T23:47:37Z

Could you please move the test/CuspCorrection/CHN to tests/molecules/CHN_ae?

anbenali · 2018-11-29T17:31:28Z

@prckent
Ref data (inputs and outputs) is always provided in the qmc-ref directory. Calculations were run for significantly more than 100X. Ref data from qmca is in the CMAKELIST.txt at the top of the file (commented). short run value is also provided in the qmc-ref directory.
Difference between ref run and short run is:
abenali@abenali:~/Work/Development/QMCPACK-Benali/qmcpack/tests/CuspCorrection/CHN/qmc-ref$ qmca -q ev *.scalar.dat

                        LocalEnergy               Variance           ratio

vmc-ref series 0 -93.339342 +/- 0.000215 2.379711 +/- 0.004316 0.0255
vmc_short-ref series 0 -93.319389 +/- 0.010942 2.320352 +/- 0.092366 0.0249

Are you refering to something else?

(Removed the Walker=1)

ye-luo · 2018-11-29T17:35:29Z

@anbenali Please keep reference input untouched and only change the input file for tests.

jtkrogel · 2018-11-29T17:55:39Z

@anbenali
Reference value and error bar should both come from the longest reference run.

Reference error bar for short test can be calculated from the long reference run by rescaling: e.g. if reference run is 100x longer than short, then reference error bar for short test is: 0.000215*sqrt(100+1) = 0.00216.

Reason is basically that short runs produce noisy error bars and thus tests fail with actual frequency deviating from 3 sigma.

Apologies if this is what you did already, your reference of 0.011 looks like it was taken directly from a single short run.

jtkrogel · 2018-11-29T18:03:13Z

@anbenali I also notice that your long reference run uses 200 blocks. This will also create a variable errorbar. For generation of reference error bar many more blocks is desired; use 10000 instead (and fewer steps of course). This will reduce the error bar of the error bar by a lot and also improve estimates of the autocorrelation time.

anbenali · 2018-11-29T19:46:58Z

@jtkrogel

For the error bar, the 100x longer or shorter is the wrong way (or at least the unpractical way) to make a test. This requirement would make sens if we have all the time we need... but the hard restriction is not the number of steps but the 30 seconds it needs to run on a given machine... hence putting the error bar raw from the run I used. If I were to use the method you give, the test will always fail since outside the 3sigma... If now I base my ref run on the 30 seconds run then my answer for the reference will have such a large error bar that we would never be able to catch an error...

Regarding the reference containing 200 blocks, this was the past requirement to avoid long files in the repo. got my previous PR blocked by @ye-luo for doing what you are suggesting. can we have a consensus here?

jtkrogel · 2018-11-29T20:15:15Z

The committed example "ref data" can be short, but the number of blocks actually used to create the reference data and the ref xml file must have more blocks. I usually include scalar.dat files from a run w/ lower blocks (typically 1000) in the repo and add a README file documenting all the reference data (optional?).

jtkrogel · 2018-11-29T21:32:54Z

@anbenali phone sometime? Better discussed interactively than through comments here.

prckent · 2018-12-04T16:18:07Z

What was the outcome of the discussions? Merge of this PR is officially pending some simple updates currently. We could choose to require them in a later PR (required for the release).

jtkrogel · 2018-12-04T16:29:28Z

Anouar is making a new test based on reference data from a 10000 blocks run. IMO we could accept this PR and make an issue relating to the test with another PR anticipated soon. Up to @anbenali.

ye-luo

Please move the test and connect it.
The current tests are not connected.

into CuspCorrection_SOA

anbenali · 2018-12-04T23:16:49Z

Hopefully the changes will please all parties..

ye-luo · 2018-12-04T23:20:28Z

tests/CMakeLists.txt

@@ -43,4 +43,5 @@ else()
    SUBDIRS("heg/heg_54_J2rpa")
    SUBDIRS("molecules")
    SUBDIRS("solids")
+    SUBDIRS("CuspCorrection")


No more need this

ye-luo

The short tests are still not stable.

Reader and saver of Cusp coeffs

d2d236d

ghost assigned anbenali Nov 26, 2018

ghost added the in progress label Nov 26, 2018

prckent requested changes Nov 26, 2018

View reviewed changes

ye-luo reviewed Nov 26, 2018

View reviewed changes

src/QMCWaveFunctions/lcao/LCAOrbitalBuilder.cpp Outdated Show resolved Hide resolved

ye-luo reviewed Nov 26, 2018

View reviewed changes

src/QMCWaveFunctions/lcao/LCAOrbitalBuilder.cpp Outdated Show resolved Hide resolved

ye-luo reviewed Nov 26, 2018

View reviewed changes

src/QMCWaveFunctions/lcao/LCAOrbitalBuilder.cpp Outdated Show resolved Hide resolved

ye-luo reviewed Nov 26, 2018

View reviewed changes

Missing header file and fixing regressions

77a8c9a

markdewing added 2 commits November 28, 2018 10:56

Remove unused code in CuspCorrection

efd0c62

Remove unused output code.

30724af

Add some documentation for the cusp correction

facfb93

Also remove some unused code.

ghost assigned markdewing Nov 28, 2018

anbenali added 3 commits November 28, 2018 17:35

Test for cuspcorrection evaluating VMC energy and not Cusp parameters

b2afdfa

remove unused files

5472d9a

Merge branch 'CuspCorrection_SOA' of https://github.com/anbenali/qmcpack

50ec4ac

into CuspCorrection_SOA

anbenali changed the title ~~[WIP] Cusp Correction for SOA - Serial~~ Cusp Correction for SOA - Serial Nov 29, 2018

remove Walker tag in vmc files

f8aa5bd

Delete tags from VMC block

ae65053

prckent approved these changes Dec 4, 2018

View reviewed changes

Merge branch 'develop' into CuspCorrection_SOA

9efe5a1

ghost assigned prckent Dec 4, 2018

ye-luo requested changes Dec 4, 2018

View reviewed changes

anbenali added 5 commits December 4, 2018 17:11

Moving Cusp test to molecule and updating data for reference

e68cba9

Moving Cusp test to molecule and updating data for reference

c78632f

remove random h5 dump

789cfd5

Merge branch 'CuspCorrection_SOA' of https://github.com/anbenali/qmcpack

22bd9de

into CuspCorrection_SOA

bad path..

25896c7

ye-luo reviewed Dec 4, 2018

View reviewed changes

ye-luo added 2 commits December 4, 2018 20:30

Fix tests/CMakeLists.txt

7c4c0ce

Merge remote-tracking branch 'github/develop' into CuspCorrection_SOA

b05d68f

ghost assigned ye-luo Dec 5, 2018

ye-luo approved these changes Dec 5, 2018

View reviewed changes

ye-luo merged commit 22cf636 into QMCPACK:develop Dec 5, 2018

ghost removed the in progress label Dec 5, 2018

anbenali deleted the CuspCorrection_SOA branch January 14, 2019 21:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cusp Correction for SOA - Serial #1172

Cusp Correction for SOA - Serial #1172

anbenali commented Nov 26, 2018 •

edited by prckent

Loading

qmc-robot commented Nov 26, 2018

qmc-robot commented Nov 26, 2018

prckent left a comment

ye-luo Nov 26, 2018

anbenali Nov 26, 2018

ye-luo Nov 26, 2018

ye-luo commented Nov 26, 2018

prckent commented Nov 26, 2018

prckent commented Nov 26, 2018

anbenali commented Nov 26, 2018

markdewing commented Nov 26, 2018

prckent commented Nov 26, 2018

anbenali commented Nov 26, 2018

prckent commented Nov 26, 2018

prckent commented Nov 28, 2018

markdewing commented Nov 28, 2018

ye-luo commented Nov 28, 2018

anbenali commented Nov 29, 2018

ye-luo commented Nov 29, 2018

jtkrogel commented Nov 29, 2018

jtkrogel commented Nov 29, 2018

anbenali commented Nov 29, 2018 •

edited

Loading

jtkrogel commented Nov 29, 2018

jtkrogel commented Nov 29, 2018

prckent commented Dec 4, 2018

jtkrogel commented Dec 4, 2018

ye-luo left a comment

anbenali commented Dec 4, 2018

ye-luo Dec 4, 2018

ye-luo left a comment •

edited

Loading

Cusp Correction for SOA - Serial #1172

Cusp Correction for SOA - Serial #1172

Conversation

anbenali commented Nov 26, 2018 • edited by prckent Loading

qmc-robot commented Nov 26, 2018

qmc-robot commented Nov 26, 2018

prckent left a comment

Choose a reason for hiding this comment

ye-luo Nov 26, 2018

Choose a reason for hiding this comment

anbenali Nov 26, 2018

Choose a reason for hiding this comment

ye-luo Nov 26, 2018

Choose a reason for hiding this comment

ye-luo commented Nov 26, 2018

prckent commented Nov 26, 2018

prckent commented Nov 26, 2018

anbenali commented Nov 26, 2018

markdewing commented Nov 26, 2018

prckent commented Nov 26, 2018

anbenali commented Nov 26, 2018

prckent commented Nov 26, 2018

prckent commented Nov 28, 2018

markdewing commented Nov 28, 2018

ye-luo commented Nov 28, 2018

anbenali commented Nov 29, 2018

ye-luo commented Nov 29, 2018

jtkrogel commented Nov 29, 2018

jtkrogel commented Nov 29, 2018

anbenali commented Nov 29, 2018 • edited Loading

jtkrogel commented Nov 29, 2018

jtkrogel commented Nov 29, 2018

prckent commented Dec 4, 2018

jtkrogel commented Dec 4, 2018

ye-luo left a comment

Choose a reason for hiding this comment

anbenali commented Dec 4, 2018

ye-luo Dec 4, 2018

Choose a reason for hiding this comment

ye-luo left a comment • edited Loading

Choose a reason for hiding this comment

anbenali commented Nov 26, 2018 •

edited by prckent

Loading

anbenali commented Nov 29, 2018 •

edited

Loading

ye-luo left a comment •

edited

Loading