New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cusp Correction for SOA - Serial #1172

Merged
merged 18 commits into from Dec 5, 2018

Conversation

Projects
None yet
6 participants
@anbenali
Copy link
Contributor

anbenali commented Nov 26, 2018

  • Cleaned print statments from Mark
  • Store xml file with Coefficients
  • Generate Coefficients if not found
  • Serial implementation for now

closes #1150

@qmc-robot

This comment has been minimized.

Copy link
Collaborator

qmc-robot commented Nov 26, 2018

Can one of the maintainers verify this patch?

1 similar comment
@qmc-robot

This comment has been minimized.

Copy link
Collaborator

qmc-robot commented Nov 26, 2018

Can one of the maintainers verify this patch?

@prckent
Copy link
Contributor

prckent left a comment

Thanks. I know that this is currently a WIP, but clearly doxygen comments are needed for all these functions. Additional comments throughout would be beneficial.

e.g. What are the magic numbers in getOneIdealLocalEnergy and is it significant that they are only there is 6 s.f.?

@@ -103,8 +103,8 @@ namespace qmcplusplus
}


LCAOrbitalBuilder::LCAOrbitalBuilder(ParticleSet& els, ParticleSet& ions, Communicate *comm, xmlNodePtr cur)
: SPOSetBuilder(comm), targetPtcl(els), sourcePtcl(ions), myBasisSet(nullptr), h5_path(""), doCuspCorrection(false)
LCAOrbitalBuilder::LCAOrbitalBuilder(ParticleSet& els, ParticleSet& ions, xmlNodePtr cur)

This comment has been minimized.

@ye-luo

ye-luo Nov 26, 2018

Contributor

This is a regression.

This comment has been minimized.

@anbenali

anbenali Nov 26, 2018

Contributor

Fixed

@@ -516,32 +670,28 @@ namespace qmcplusplus
LCAOrbitalSet *lcos = nullptr;
LCAOrbitalSetWithCorrection *lcwc = nullptr;
if (doCuspCorrection) {
lcwc =new LCAOrbitalSetWithCorrection(sourcePtcl, targetPtcl, myBasisSet, rank()==0);
lcwc =new LCAOrbitalSetWithCorrection(sourcePtcl, targetPtcl, myBasisSet, ReportLevel);

This comment has been minimized.

@ye-luo

ye-luo Nov 26, 2018

Contributor

This is a regression.

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Nov 26, 2018

In the mixed precision case, we probably need to do the cusp correction construction in full precision while do the evaluation in single. Do we have a plan how to do this?

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Nov 26, 2018

Re: mixed-precision. I am also interested in the plan, but for an initial implementation I wouldn't object if we only supported double. A working executable path is needed in SoA to help debug #1145 .

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Nov 26, 2018

OK to test

@anbenali

This comment has been minimized.

Copy link
Contributor

anbenali commented Nov 26, 2018

Keep in Mind that the point is to have this available for now for anyone "stuck with a bug" and also to help find errors if any.

I am making the PR because I needed this code, but all the credit goes to @markdewing who made 99.99% of the work. (which means I cannot make the doc for the code)

@markdewing

This comment has been minimized.

Copy link
Contributor

markdewing commented Nov 26, 2018

I will write some documentation for these functions.

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Nov 26, 2018

@anbenali

This comment has been minimized.

Copy link
Contributor

anbenali commented Nov 26, 2018

@prckent I am temptd to stage this in 2 separate PRs. one for Serial and one for parallel implementation just because of the urgency of the matter. I am testing the code on a very large set of molecules and in some cases there is significant drop in the variance and multidet is slightly stabilized from large fluctuations.
If that is fine with you I rather do it that way. Otherwise it will be one more week I guess before I can touch the parallel part..

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Nov 26, 2018

@anbenali I assumed this was your plan from the PR name. When the serial implementation is in we can already begin to learn about #1145 and other surprises. Running faster comes later.

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Nov 28, 2018

@markdewing Do you plan to update the code in this PR with some comments, or should we merge now and update later?
@anbenali is this ready to merge apart from the docs? Remove the WIP if so.

Add some documentation for the cusp correction
Also remove some unused code.
@markdewing

This comment has been minimized.

Copy link
Contributor

markdewing commented Nov 28, 2018

I'm working on some comments, documentation, and removing some unused code. I can probably get that into this branch today.

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Nov 28, 2018

Could you please move the test/CuspCorrection/CHN to tests/molecules/CHN_ae?

@anbenali

This comment has been minimized.

Copy link
Contributor

anbenali commented Nov 29, 2018

@prckent
Ref data (inputs and outputs) is always provided in the qmc-ref directory. Calculations were run for significantly more than 100X. Ref data from qmca is in the CMAKELIST.txt at the top of the file (commented). short run value is also provided in the qmc-ref directory.
Difference between ref run and short run is:
abenali@abenali:~/Work/Development/QMCPACK-Benali/qmcpack/tests/CuspCorrection/CHN/qmc-ref$ qmca -q ev *.scalar.dat

                        LocalEnergy               Variance           ratio 

vmc-ref series 0 -93.339342 +/- 0.000215 2.379711 +/- 0.004316 0.0255
vmc_short-ref series 0 -93.319389 +/- 0.010942 2.320352 +/- 0.092366 0.0249

Are you refering to something else?

(Removed the Walker=1)

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Nov 29, 2018

@anbenali Please keep reference input untouched and only change the input file for tests.

@jtkrogel

This comment has been minimized.

Copy link
Contributor

jtkrogel commented Nov 29, 2018

@anbenali
Reference value and error bar should both come from the longest reference run.

Reference error bar for short test can be calculated from the long reference run by rescaling: e.g. if reference run is 100x longer than short, then reference error bar for short test is: 0.000215*sqrt(100+1) = 0.00216.

Reason is basically that short runs produce noisy error bars and thus tests fail with actual frequency deviating from 3 sigma.

Apologies if this is what you did already, your reference of 0.011 looks like it was taken directly from a single short run.

@jtkrogel

This comment has been minimized.

Copy link
Contributor

jtkrogel commented Nov 29, 2018

@anbenali I also notice that your long reference run uses 200 blocks. This will also create a variable errorbar. For generation of reference error bar many more blocks is desired; use 10000 instead (and fewer steps of course). This will reduce the error bar of the error bar by a lot and also improve estimates of the autocorrelation time.

@anbenali

This comment has been minimized.

Copy link
Contributor

anbenali commented Nov 29, 2018

@jtkrogel

For the error bar, the 100x longer or shorter is the wrong way (or at least the unpractical way) to make a test. This requirement would make sens if we have all the time we need... but the hard restriction is not the number of steps but the 30 seconds it needs to run on a given machine... hence putting the error bar raw from the run I used. If I were to use the method you give, the test will always fail since outside the 3sigma... If now I base my ref run on the 30 seconds run then my answer for the reference will have such a large error bar that we would never be able to catch an error...

Regarding the reference containing 200 blocks, this was the past requirement to avoid long files in the repo. got my previous PR blocked by @ye-luo for doing what you are suggesting. can we have a consensus here?

@jtkrogel

This comment has been minimized.

Copy link
Contributor

jtkrogel commented Nov 29, 2018

The committed example "ref data" can be short, but the number of blocks actually used to create the reference data and the ref xml file must have more blocks. I usually include scalar.dat files from a run w/ lower blocks (typically 1000) in the repo and add a README file documenting all the reference data (optional?).

@jtkrogel

This comment has been minimized.

Copy link
Contributor

jtkrogel commented Nov 29, 2018

@anbenali phone sometime? Better discussed interactively than through comments here.

@prckent

This comment has been minimized.

Copy link
Contributor

prckent commented Dec 4, 2018

What was the outcome of the discussions? Merge of this PR is officially pending some simple updates currently. We could choose to require them in a later PR (required for the release).

@jtkrogel

This comment has been minimized.

Copy link
Contributor

jtkrogel commented Dec 4, 2018

Anouar is making a new test based on reference data from a 10000 blocks run. IMO we could accept this PR and make an issue relating to the test with another PR anticipated soon. Up to @anbenali.

@prckent

prckent approved these changes Dec 4, 2018

@ye-luo
Copy link
Contributor

ye-luo left a comment

Please move the test and connect it.
The current tests are not connected.

@anbenali

This comment has been minimized.

Copy link
Contributor

anbenali commented Dec 4, 2018

Hopefully the changes will please all parties..

@@ -43,4 +43,5 @@ else()
SUBDIRS("heg/heg_54_J2rpa")
SUBDIRS("molecules")
SUBDIRS("solids")
SUBDIRS("CuspCorrection")

This comment has been minimized.

@ye-luo

ye-luo Dec 4, 2018

Contributor

No more need this

@ye-luo

ye-luo approved these changes Dec 5, 2018

Copy link
Contributor

ye-luo left a comment

The short tests are still not stable.

@ye-luo ye-luo merged commit 22cf636 into QMCPACK:develop Dec 5, 2018

2 checks passed

qmcpack rhea Build finished.
Details
qmcpack rhea (gpu) Build finished.
Details

@wafflebot wafflebot bot removed the in progress label Dec 5, 2018

@anbenali anbenali deleted the anbenali:CuspCorrection_SOA branch Jan 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment