Remaining tasks for AoS to SoA transition

[ye-luo]

This is the final list of tasks that have priority before the remaining AoS code is fully removed from the source. The AoS removal likely will occur sometime in Oct. 2019. Even code listed as "priority" below will be removed once the deadline is exceeded if the assignee(s) have not ported by then and this issue will be closed.

All remaining AoS code has been encapsulated by if-defs. The changes made in the following PR highlight all remaining AoS code:

https://github.com/QMCPACK/qmcpack/pull/1846/files

Priority code to port before the Oct. deadline:

• g(r) (issue: gofr estimator gives incorrect results for SoA #1699, assignee: @markdewing, PR: Fix g(r) for SoA #2035)
• Chiesa/Ceperley and bare force estimators (issue: SoA ports needed for Chiesa/Ceperley and Bare force estimators #1795, assignee: , PR: )
• energy density and energy density matrix estimators for EKT (issue: SoA ports needed for energy density and energy density matrix #1796, assignee: @jtkrogel, PR: Transition single particle energy evaluation to SoA #1869)
• orbital optimization (issue: , assignee: @spinedaflores , PR: Wrapper Class RotationHelper for Orbital Optimization  #1893)
• lattice deviation estimator (issue: , assignee: @Paul-St-Young, PR: SOA versions of latdev and ionwf #1938)
• "Chiesa" S(k) kinetic energy correction (issue: Chiesa finite size correction not compatible with SoA #2025, assignee: @Hyeondeok-Shin @ye-luo, PR: Enable Chiesa kinetic energy correction on SoA build & add deterministic test for KEcorr #2050).
• NLPP derivatives in optimization (issue: NLPP derivatives in optimization need SoA #2083, assignee: @ye-luo, PR Fix NLPP derivative in SoA #2179)

Code that is slated for deletion after the Oct. deadline (volunteers welcome!):

• complex HEG backflow (see HEG Backflow Test Fails with Complex SoA Build #1227)
• spin-dependent electron-ion backflow
• AoS cusp correction code (SoA code has problems, see Cusp Correction AoS to SoA inconsistency (leading to "crazy" variance with SoA)  #1388)
• FDLR wavefunction
• numerical orbitals
• stress estimator (see Stress not OK with SoA #970)
• core polarization potential

Note: tests that pass for AoS but fail for SoA:

• long-diamondC_1x1x1_pp-vmc_sdj-1-16-flux (wavefunction error!)
• short-c_no-hf_vmc-1-16 (numerical orbitals)
• long-c_no-hf_vmc-1-16
• short-c_no-sj_dmc-1-16
• long-c_no-sj_dmc-1-16
• short-H2-FDLR-16-1 (FDLR wavefunction)
• short-H2-orb-opt-16-1 (orbital optimization)
• short-H4-orb-opt-dmc-1-16

[PDoakORNL]

What about the optimizable wave function support?

I'm working on the CUDA code now.
I've begun JastrowCUDAClass switch to the OrbitalBase base class.

[ye-luo]

JastrowCUDAClass switch to the OrbitalBase base class what is this?
Currently, the code does

class OneBodyJastrowOrbitalBspline :
  public OneBodyJastrowOrbital<BsplineFunctor<OrbitalBase::RealType> >

OneBodyJastrowOrbital needs to be replaced with J1OrbitalSoA.

Jastrow optimization is supported by the CUDA build.

[PDoakORNL]

The optimization I was talking about was of the single particle orbitals.

At the moment I have a J1OrbitalCUDASoA so as to look at what can be shared and what can't.
Not sure if it will be
J1OrbitalCUDASoA : public J1OrbitalSoA and eventually J1OrbitalCUDA : public J1Orbital
or
J1OrbitalCUDA : public J1OrbitalBase
J1OrbitalCPU : public J1OrbitalBase

So that we end up with a call like this:

return createOneBodyJastrow<J1OrbitalCUDASoA<RadFuncType>,DiffOneBodySpinJastrowOrbital<RadFuncType> >(cur);

A certain amount of the old CUDA OneBody and TwoBody code will go in but my current strategy is to make sure that any feature we don't have in CUDA we are following the SoA "method" and can degrade to the CPU implementation.

Although this is speculative at the moment since I'm still working in the determinant code.

[prckent]

Quick comment on the CUDA-> SoA transition, agreeing with @ye-luo 's list at the top: happily the CUDA SoA build is working for solids apart from runs involving Jastrows. e.g. https://cdash.qmcpack.org/CDash/testDetails.php?test=2572921&build=24790 shows that DMC is working with single determinant no Jastrow. After that the gaps are things like J3 and the new hybrid rep.

[PDoakORNL]

RPAJastro is dependent on TwoBodyJastroOrbital. It also looks a bit unfinished in general.

[jtkrogel]

@ye-luo, please add this to the checklist at the top: "evaluateHessian for J1 and J2". These are present in AoS, but not SoA and I am using them in research now.

[jtkrogel]

@ye-luo please also add the evaluate_sp functions that appear in a number of QMCHamiltonianBase. These functions are needed to obtain and transmit single particle energies that are used by Traces, energy density, and energy density matrix.

[jtkrogel]

Please also add Stress, see #970.

[prckent]

Note: We intend to flip the default to SOA shortly, since nearly all common features are implemented and we have observed that some users are overlooking the performance benefits of the SOA code. If anything critical is missing, please update this issue and the checklist at the top.

Current gating functionalities are the derivatives for backflow.

[prckent]

Update: SoA is now the default, but this issue remains relevant: at some point the AoS code will simply be deleted along with the legacy CUDA code. This will restore QMCPACK to having a single implementation "fork". To access functionality not ported will then require running an old version of QMCPACK or first porting the functionality to the current version.

[jtkrogel]

@spinedaflores Please can you comment here about your timeline for porting the orbital optimization code over?

@Paul-St-Young Is the lattice deviation estimator important to you? If so, can you do the port to SoA?

[spinedaflores]

I'll aim to have my pull request ready by next Friday, but hopefully will have it done before then.

[jtkrogel]

@spinedaflores Thanks. You are definitely on track to beat the rest of us.

[Paul-St-Young]

@jtkrogel yes, I would like to keep the lattice deviation estimator working. I will look into porting

[markdewing]

I'm thinking about putting #ifndef ENABLE_SOA around the distance table elements that are only for AoS (M, J, df, rinv, etc.) and around blocks of code that use these members. This would make the AoS sections of code more visible, and clearer when it is time to delete them.

[jtkrogel]

I think this is a good idea.

[prckent]

Chiesa/Ceperley and bare force estimators are currently marked as done, but #2025 indicates they are not. Should we consider this task done or not?

[jtkrogel]

Marked as done following discussion with @tiihonej. @tiihonej and/or @rcclay should comment here.

[tiihonej]

@jtkrogel Yes, Chiesa/Ceperley and bare force estimators have been ported to SoA. @prckent, I thought #2025 refers to a different estimator with whose details I'm not familiar.

[prckent]

Calculation of pp derivatives for optimization needs to be captured above. It is a critical capability.

[jtkrogel]

@prckent Issues have been added/updated.

[prckent]

All the items on the task list are now completed. #2179 was the last item. Following the v3.9.0 release, removal of the AoS code will commence.

[jtkrogel]

Basically done?

[prckent]

Agreed. AoS is no longer a build option.