Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nexus: enable improved generation of QMCPACK inputs #1471

Merged
merged 17 commits into from Mar 28, 2019

Conversation

Projects
None yet
4 participants
@jtkrogel
Copy link
Contributor

jtkrogel commented Mar 22, 2019

This PR pulls in QMCPACK input generation methods from qmcpack_worfklows into the main code (now made accessible via generate_qmcpack). This update is fully backwards compatible with existing Nexus scripts, as confirmed by passing tests for unchanged example inputs.

The main advantages of the changes are to enable more compact user input to request sensible defaults for initial Jastrow factors and OPT/VMC/DMC calculation sections.

For example, good default Jastrow factors are generated by setting, e.g. J2=True or J3=True in generate_qmcpack rather than providing more detailed inputs via the jastrows keyword:

Input:
generate_qmcpack(...,J3=True)

Output:
         <jastrow type="One-Body" name="J1" function="bspline" source="ion0" print="yes">
            <correlation elementType="C" size="10" rcut="4.7703702464" cusp="0.0">
               <coefficients id="eC" type="Array">                  
0 0 0 0 0 0 0 0 0 0
               </coefficients>
            </correlation>
         </jastrow>
         <jastrow type="Two-Body" name="J2" function="bspline" print="yes">
            <correlation speciesA="u" speciesB="u" size="10" rcut="4.7703702464">
               <coefficients id="uu" type="Array">                  
0 0 0 0 0 0 0 0 0 0
               </coefficients>
            </correlation>
            <correlation speciesA="u" speciesB="d" size="10" rcut="4.7703702464">
               <coefficients id="ud" type="Array">                  
0 0 0 0 0 0 0 0 0 0
               </coefficients>
            </correlation>
         </jastrow>
         <jastrow type="eeI" name="J3" function="polynomial" print="yes" source="ion0">
            <correlation ispecies="C" especies1="u" especies2="u" isize="3" esize="3" rcut="4.7703702464">
               <coefficients id="uuC" type="Array" optimize="yes"/>
            </correlation>
            <correlation ispecies="C" especies1="u" especies2="d" isize="3" esize="3" rcut="4.7703702464">
               <coefficients id="udC" type="Array" optimize="yes"/>
            </correlation>
         </jastrow>

The default cutoffs for J1/J2 are the n.n. wigner radius and the number of B-spline coeffecients is one per 0.5 Bohr.

QMC sections are autogenerated by setting qmc='XXX' instead of providing structured input via calculations. Examples for optimization (quartic/oneshiftonly), VMC, and DMC (direct and timestep extrapolation) are shown below.

Optimization:

Input:
generate_qmcpack(...,qmc='opt',minmethod='quartic',cost='energy')

Output:
   <loop max="4">
      <qmc method="linear" move="pbyp" checkpoint="-1">
         <cost name="energy"              >    0.0                </cost>
         <cost name="unreweightedvariance">    1.0                </cost>
         <cost name="reweightedvariance"  >    0.0                </cost>
         <parameter name="warmupSteps"         >    300                </parameter>
         <parameter name="blocks"              >    100                </parameter>
         <parameter name="steps"               >    1                  </parameter>
         <parameter name="subSteps"            >    10                 </parameter>
         <parameter name="timestep"            >    0.3                </parameter>
         <parameter name="useDrift"            >    no                 </parameter>
         <parameter name="samples"             >    204800             </parameter>
         <parameter name="MinMethod"           >    quartic            </parameter>
         <parameter name="minwalkers"          >    0.3                </parameter>
         <parameter name="nonlocalpp"          >    yes                </parameter>
         <parameter name="useBuffer"           >    yes                </parameter>
         <parameter name="alloweddifference"   >    0.0001             </parameter>
         <parameter name="exp0"                >    -6                 </parameter>
         <parameter name="bigchange"           >    10.0               </parameter>
         <parameter name="stepsize"            >    0.15               </parameter>
         <parameter name="nstabilizers"        >    1                  </parameter>
         <estimator name="LocalEnergy" hdf5="yes"/>
      </qmc>
   </loop>
   <loop max="12">
      <qmc method="linear" move="pbyp" checkpoint="-1">
         <cost name="energy"              >    1.0                </cost>
         <cost name="unreweightedvariance">    0.0                </cost>
         <cost name="reweightedvariance"  >    0.0                </cost>
         <parameter name="warmupSteps"         >    300                </parameter>
         <parameter name="blocks"              >    100                </parameter>
         <parameter name="steps"               >    1                  </parameter>
         <parameter name="subSteps"            >    10                 </parameter>
         <parameter name="timestep"            >    0.3                </parameter>
         <parameter name="useDrift"            >    no                 </parameter>
         <parameter name="samples"             >    204800             </parameter>
         <parameter name="MinMethod"           >    quartic            </parameter>
         <parameter name="minwalkers"          >    0.3                </parameter>
         <parameter name="nonlocalpp"          >    yes                </parameter>
         <parameter name="useBuffer"           >    yes                </parameter>
         <parameter name="alloweddifference"   >    0.0001             </parameter>
         <parameter name="exp0"                >    -6                 </parameter>
         <parameter name="bigchange"           >    10.0               </parameter>
         <parameter name="stepsize"            >    0.15               </parameter>
         <parameter name="nstabilizers"        >    1                  </parameter>
         <estimator name="LocalEnergy" hdf5="yes"/>
      </qmc>
   </loop>
Input:
generate_qmcpack(...,qmc='opt',minmethod='oneshiftonly')

Output:
   <loop max="12">
      <qmc method="linear" move="pbyp" checkpoint="-1">
         <cost name="energy"              >    0.0                </cost>
         <cost name="unreweightedvariance">    1.0                </cost>
         <cost name="reweightedvariance"  >    0.0                </cost>
         <parameter name="warmupSteps"         >    300                </parameter>
         <parameter name="blocks"              >    100                </parameter>
         <parameter name="steps"               >    1                  </parameter>
         <parameter name="subSteps"            >    10                 </parameter>
         <parameter name="timestep"            >    0.3                </parameter>
         <parameter name="useDrift"            >    no                 </parameter>
         <parameter name="samples"             >    204800             </parameter>
         <parameter name="MinMethod"           >    OneShiftOnly         </parameter>
         <parameter name="minwalkers"          >    0.3                </parameter>
         <parameter name="nonlocalpp"          >    yes                </parameter>
         <parameter name="shift_i"             >    0.01               </parameter>
         <parameter name="shift_s"             >    1.0                </parameter>
         <estimator name="LocalEnergy" hdf5="yes"/>
      </qmc>
   </loop>

VMC:

Input:
generate_qmcpack(...,qmc='vmc')

Output:
   <qmc method="vmc" move="pbyp" checkpoint="-1">
      <parameter name="walkers"             >    1               </parameter>
      <parameter name="warmupSteps"         >    50              </parameter>
      <parameter name="blocks"              >    800             </parameter>
      <parameter name="steps"               >    10              </parameter>
      <parameter name="subSteps"            >    3               </parameter>
      <parameter name="timestep"            >    0.3             </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>

DMC:

Input:
generate_qmcpack(...,qmc='dmc')

Output:
   <qmc method="vmc" move="pbyp" checkpoint="-1">
      <parameter name="walkers"             >    1               </parameter>
      <parameter name="warmupSteps"         >    30              </parameter>
      <parameter name="blocks"              >    40              </parameter>
      <parameter name="steps"               >    10              </parameter>
      <parameter name="subSteps"            >    3               </parameter>
      <parameter name="timestep"            >    0.3             </parameter>
      <parameter name="samples"             >    2048            </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>
   <qmc method="dmc" move="pbyp" checkpoint="-1">
      <parameter name="warmupSteps"         >    20              </parameter>
      <parameter name="blocks"              >    200             </parameter>
      <parameter name="steps"               >    10              </parameter>
      <parameter name="timestep"            >    0.01            </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>

DMC timestep extrapolation:

Input:
generate_qmcpack(...,qmc='dmc',timestep=0.02,ntimesteps = 4)

Output:
   <qmc method="vmc" move="pbyp" checkpoint="-1">
      <parameter name="walkers"             >    1               </parameter>
      <parameter name="warmupSteps"         >    30              </parameter>
      <parameter name="blocks"              >    40              </parameter>
      <parameter name="steps"               >    10              </parameter>
      <parameter name="subSteps"            >    3               </parameter>
      <parameter name="timestep"            >    0.3             </parameter>
      <parameter name="samples"             >    2048            </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>
   <qmc method="dmc" move="pbyp" checkpoint="-1">
      <parameter name="warmupSteps"         >    20              </parameter>
      <parameter name="blocks"              >    200             </parameter>
      <parameter name="steps"               >    10              </parameter>
      <parameter name="timestep"            >    0.02            </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>
   <qmc method="dmc" move="pbyp" checkpoint="-1">
      <parameter name="warmupSteps"         >    40              </parameter>
      <parameter name="blocks"              >    200             </parameter>
      <parameter name="steps"               >    20              </parameter>
      <parameter name="timestep"            >    0.01            </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>
   <qmc method="dmc" move="pbyp" checkpoint="-1">
      <parameter name="warmupSteps"         >    80              </parameter>
      <parameter name="blocks"              >    200             </parameter>
      <parameter name="steps"               >    40              </parameter>
      <parameter name="timestep"            >    0.005           </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>
   <qmc method="dmc" move="pbyp" checkpoint="-1">
      <parameter name="warmupSteps"         >    160             </parameter>
      <parameter name="blocks"              >    200             </parameter>
      <parameter name="steps"               >    80              </parameter>
      <parameter name="timestep"            >    0.0025          </parameter>
      <estimator name="LocalEnergy" hdf5="yes"/>
   </qmc>
@markdewing

This comment has been minimized.

Copy link
Contributor

markdewing commented Mar 22, 2019

okay to test

@qmc-robot

This comment has been minimized.

Copy link
Collaborator

qmc-robot commented Mar 22, 2019

Can one of the maintainers verify this patch?

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 22, 2019

What is the main reason for yes <estimator name="LocalEnergy" hdf5="yes"/>

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 22, 2019

Please also remove in OneShiftOnly

         <cost name="energy"              >    0.0                </cost>
         <cost name="unreweightedvariance">    1.0                </cost>
         <cost name="reweightedvariance"  >    0.0                </cost>
         <parameter name="shift_i"             >    0.01               </parameter>
         <parameter name="shift_s"             >    1.0                </parameter>

Please follow the example in the manual for one shift only. Two optimization sections are recommended.

@jtkrogel

This comment has been minimized.

Copy link
Contributor Author

jtkrogel commented Mar 22, 2019

With hdf5=yes the scalar data is written to the stat.h5.

Why should the cost function be removed? It is simply user choice. The shift values are just the defaults, so I don't see why this is harmful.

I can enable a way to use separate minwalkers if desired.

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 22, 2019

  1. Minimal I/O is preferred.
  2. Changing cost function fraction doesn't affect the OneShiftOnly optimizer. It doesn't affect its decision. The default values are good enough.
  3. If paramters are the same as the default, they should not list. If the default is changed, it should be propagated instead of being blocked by the workflow setting.
@jtkrogel

This comment has been minimized.

Copy link
Contributor Author

jtkrogel commented Mar 27, 2019

I don't personally see why it's such a big issue, but I am willing to change (1) and (3) according to @ye-luo's requests.

I'm confused about your statements regarding the cost function. Is oneshift only ever optimizing according to a fixed cost function regardless of user input? If so, which is it (energy?)?

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 27, 2019

Quartic optimizer makes the decision whether a new set of parameter is accepted or ignored based on fitting the CostFunction values to a quartic function. OneShiftOnly is pure energy minimization and has a different strategy. It almost always accept new parameter set unless miniwalkers screens crazy bad parameters. CostFunction is calculated and printed only for user to have a feeling of the optimization but it doesn't get involved in any decision making of OSO.
It is a user choice in general but unnecessary for OSO. Users are not able to influence the optimizer by changing the input parameters. Thus, it is better not to put it in the input example. The default value is 0.9 energy + 0.1 reweightedvariance. Your input example has reweightedvariance = 1.0 just confuses users that it looks like a variance minimization.

@jtkrogel

This comment has been minimized.

Copy link
Contributor Author

jtkrogel commented Mar 27, 2019

OK, so OSO cost function is fixed internally regardless of user input.

Is it fixed to pure energy (1.0energy) or 0.9energy+0.1*reweightedvariance? You mention both above.

(NB: recent commits address (1) above)

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 27, 2019

The optimization has two stages.

In the first stage, the optimizer just compute WF and LocalEnergy derivatives and propose a new set of parameters. All choices of optimizer are doing linear method energy minimization rescale/quartic/OSO.
There is no cost function involved.

In the second stage,
quartic:
Compute the cost function values and fit them to a quartic function and make a decision.
OSO:
Compute the cost function value and print it. The decision making is only based on minwalkers.

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 27, 2019

@jtkrogel It is not fixed because the energy minimization doesn't even know the costfunction. user input of cost function is respected but used differently.

@jtkrogel

This comment has been minimized.

Copy link
Contributor Author

jtkrogel commented Mar 27, 2019

OK, perhaps we are conflating two things: cost function as input, and the actual target of optimization.

Does OSO find the lowest energy solution? If not energy, what is it minimizing by default? Is there (and what is) a way to influence what it is minimizing via user input?

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 27, 2019

In the first stage, OSO is targetting energy. In the second stage, OSO doesn't pursuit the absolute lowest CostFunction because each VMC section has limited samples and the energy fluctuates. The parameter set gives the lowest CostFunction is not necessarily the best parameter set. For this reason, OSO let the optimization continue while the quartic struggles with the `optimal' CostFunction. As long as stage 1 is correct, OSO should converge statistically.

@jtkrogel

This comment has been minimized.

Copy link
Contributor Author

jtkrogel commented Mar 27, 2019

See #1494 (the final result of this line of discussion via phone).

Next, I will make commits addressing points 2 and 3 above.

@jtkrogel

This comment has been minimized.

Copy link
Contributor Author

jtkrogel commented Mar 27, 2019

Points 2 and 3 have been addressed. Cost function and defaults are now suppressed for OneShiftOnly. A simple means is added to request a split loop w/ smaller and larger minwalkers as follows:

Input:
generate_qmcpack(...,qmc='opt',minmethod='oneshiftonly',init_cycles=6)

Output:
   <loop max="6">
      <qmc method="linear" move="pbyp" checkpoint="-1">
         <parameter name="warmupSteps"         >    300                </parameter>
         <parameter name="blocks"              >    100                </parameter>
         <parameter name="steps"               >    1                  </parameter>
         <parameter name="subSteps"            >    10                 </parameter>
         <parameter name="timestep"            >    0.3                </parameter>
         <parameter name="useDrift"            >    no                 </parameter>
         <parameter name="samples"             >    204800             </parameter>
         <parameter name="MinMethod"           >    OneShiftOnly         </parameter>
         <parameter name="minwalkers"          >    0.0001             </parameter>
         <parameter name="nonlocalpp"          >    yes                </parameter>
      </qmc>
   </loop>
   <loop max="12">
      <qmc method="linear" move="pbyp" checkpoint="-1">
         <parameter name="warmupSteps"         >    300                </parameter>
         <parameter name="blocks"              >    100                </parameter>
         <parameter name="steps"               >    1                  </parameter>
         <parameter name="subSteps"            >    10                 </parameter>
         <parameter name="timestep"            >    0.3                </parameter>
         <parameter name="useDrift"            >    no                 </parameter>
         <parameter name="samples"             >    204800             </parameter>
         <parameter name="MinMethod"           >    OneShiftOnly         </parameter>
         <parameter name="minwalkers"          >    0.5                </parameter>
         <parameter name="nonlocalpp"          >    yes                </parameter>
      </qmc>
   </loop>
@jtkrogel

This comment has been minimized.

Copy link
Contributor Author

jtkrogel commented Mar 27, 2019

@ye-luo OK to merge now?

@ye-luo

ye-luo approved these changes Mar 27, 2019

@ye-luo

This comment has been minimized.

Copy link
Contributor

ye-luo commented Mar 27, 2019

I feel there is some inconsistency between blocks, steps and samples. At least one parameter will be dropped. But it is a overall parameter design problem.

@ye-luo ye-luo merged commit 55ab1af into QMCPACK:develop Mar 28, 2019

2 checks passed

rhea-cpu
Details
rhea-gpu
Details

@wafflebot wafflebot bot removed the in progress label Mar 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.