The developer wish list #402

Open
dgasmith opened this Issue Jun 6, 2016 · 30 comments

Comments

Projects
None yet
7 participants
@dgasmith
Member

dgasmith commented Jun 6, 2016

This is a list of things that are generally agreed as "good things to do", but nobody has actually tackled yet. The hope is that this is used a central repository of "have you thought about this" comments. Note that this is a not a feature wish list.

High

  • Deprecate PSIO by moving to HDF5. With a temporary under the hood PSIO move to HDF5.
  • Remove boost::python and replace with PyBind11.
  • Possibly remove Boost with C++11 features. Std managed pointers are high on the list.
  • Switch Psi4 from a binary to a library
    • Allow more flexible external libraries
  • Remove C-side python calls. DFT-D3, DFT functionals, BasisSet parsing, etc.
  • Add a dictionary to the Wavefunction object thats holds the current Psi Variables.

Medium

  • Rewrite LibOptions as a property map or just a general dictionary. Needs to be more object oriented, less global, and capable of encompassing all QCDB.
  • Full Python3 support. This is mostly done, but small sections are not Python3 compatible.
  • Ability to combine DF fitting sets via partial decomposition of a expanded Coulomb metric.
  • ECP would be a great addition.
  • Purge all char* from Psi4. Issues with dropping pointers and python exportation.
  • A smarter SCF solver that can change iterations based on current conditions, see #211.

Low

  • More flexibility in compilation of integrals. For example compiling 3-index and Coulomb metric integrals at a higher AM than the conventional basis set.
  • 4th CMake rewrite. Currently overly cautious about what needs to be recompiled.
    Our CMake files should ensure that the compilers support all the features that we need.
  • EFP Gradients: psi4/psi4private#70
  • Uniform space setter for DMRG/CI/MCSCF/etc computations.
  • change_file_namespace should be able to tie multiple files together and should return the status rather than None, #645.
  • Molecule parser can accept atomic numbers instead of symbols, #418.
  • DETCI for more than 256 orbitals
  • Automatic choice between rhf/uhf/rohf/cuhf for input molecules.
  • Potenital integral derivative performance can be improved, see #3.
  • Allow a DECON keyword to basis sets #43.
  • Remove char * for std::string.

Modify, expand, delete as desired. If you take up a feature make sure to post here so that we do not duplicate effort.

@jgonthier

This comment has been minimized.

Show comment
Hide comment
@jgonthier

jgonthier Jun 6, 2016

Member

Medium

  • Parallelize the mints integral generation:
    Needs to use asynchronous I/O to have one process writing to disk while others compute. Basic technology to do that is in the new PKJK object.
  • Reduce PKJK I/O
    Preliminary experiment shows that storing only the ordered supermatrix for J and reordering it on the fly when forming K is faster than I/O for storing/reading ordered supermatrix for K.
  • Sieve PKJK
    Currently PKJK only uses sieving for integral computations. Could use it to only I/O significant integrals. We would probably need a specialized iterator that goes only through significant integrals and gives us back the absolute indices for building J and K.
Member

jgonthier commented Jun 6, 2016

Medium

  • Parallelize the mints integral generation:
    Needs to use asynchronous I/O to have one process writing to disk while others compute. Basic technology to do that is in the new PKJK object.
  • Reduce PKJK I/O
    Preliminary experiment shows that storing only the ordered supermatrix for J and reordering it on the fly when forming K is faster than I/O for storing/reading ordered supermatrix for K.
  • Sieve PKJK
    Currently PKJK only uses sieving for integral computations. Could use it to only I/O significant integrals. We would probably need a specialized iterator that goes only through significant integrals and gives us back the absolute indices for building J and K.
@jgonthier

This comment has been minimized.

Show comment
Hide comment
@jgonthier

jgonthier Jul 5, 2016

Member

Medium

  • Design and implement a memory manager. It should keep track of the memory allocated and provide the user with the currently available memory.
Member

jgonthier commented Jul 5, 2016

Medium

  • Design and implement a memory manager. It should keep track of the memory allocated and provide the user with the currently available memory.
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Jul 5, 2016

Member

Purportedly, @bennybp has a memory manager in pulsar.

Member

loriab commented Jul 5, 2016

Purportedly, @bennybp has a memory manager in pulsar.

@dgasmith dgasmith referenced this issue Jul 26, 2016

Closed

Boost removal #450

37 of 37 tasks complete
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Oct 5, 2016

Member

Moved from #476 so as to not clutter issues.

  • Here lie uses for general Cholesky machinery.
    • Fock-type terms in DF-SAPT, particularly higher-order, see #317
    • If you care about 6-zeta accuracy, you probably oughtn't to be using density fitting. Nevertheless, having orbital basis sets for which no auxiliary is appropriate (even as overkill, like QZ for sto-3g) is troubling for the auxiliary defaulting machinery. On-the-fly construction could help. Maybe a more plausible case is when auxiliary basis sets aren't available for a particular element or for tight functions.

Moreover, Susi Lehtola comments:

Well, you could do compact atomic Cholesky to construct a fitting basis for an arbitrary orbital basis set on-the-fly. [doi 10.1063/1.3116784]

Member

loriab commented Oct 5, 2016

Moved from #476 so as to not clutter issues.

  • Here lie uses for general Cholesky machinery.
    • Fock-type terms in DF-SAPT, particularly higher-order, see #317
    • If you care about 6-zeta accuracy, you probably oughtn't to be using density fitting. Nevertheless, having orbital basis sets for which no auxiliary is appropriate (even as overkill, like QZ for sto-3g) is troubling for the auxiliary defaulting machinery. On-the-fly construction could help. Maybe a more plausible case is when auxiliary basis sets aren't available for a particular element or for tight functions.

Moreover, Susi Lehtola comments:

Well, you could do compact atomic Cholesky to construct a fitting basis for an arbitrary orbital basis set on-the-fly. [doi 10.1063/1.3116784]

@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Nov 16, 2016

Member

@CDSherrill requests a disk use estimate. Could just be final disk use as a system call before cleaning (https://github.com/psi4/psi4/blob/master/psi4/run_psi4.py#L196).

Member

loriab commented Nov 16, 2016

@CDSherrill requests a disk use estimate. Could just be final disk use as a system call before cleaning (https://github.com/psi4/psi4/blob/master/psi4/run_psi4.py#L196).

@CDSherrill

This comment has been minimized.

Show comment
Hide comment
@CDSherrill

CDSherrill Nov 16, 2016

Member

Yes, this would be very helpful so that users will know how much disk is
needed (even approximately) for subsequent jobs of this type. It is also
helpful to people planning to buy new hardware and who need to know how
much disk to buy.... one can look back through old outputs on some of the
challenging computations and see how much disk they used.

On Wed, Nov 16, 2016 at 3:06 PM, Lori A. Burns notifications@github.com
wrote:

@CDSherrill https://github.com/CDSherrill requests a disk use estimate.
Could just be final disk use as a system call before cleaning (
https://github.com/psi4/psi4/blob/master/psi4/run_psi4.py#L196).


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#402 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AC9QdgSg1IZHU-ezlwNRXjWKX3oEMSQaks5q-2JfgaJpZM4IvHrd
.

Member

CDSherrill commented Nov 16, 2016

Yes, this would be very helpful so that users will know how much disk is
needed (even approximately) for subsequent jobs of this type. It is also
helpful to people planning to buy new hardware and who need to know how
much disk to buy.... one can look back through old outputs on some of the
challenging computations and see how much disk they used.

On Wed, Nov 16, 2016 at 3:06 PM, Lori A. Burns notifications@github.com
wrote:

@CDSherrill https://github.com/CDSherrill requests a disk use estimate.
Could just be final disk use as a system call before cleaning (
https://github.com/psi4/psi4/blob/master/psi4/run_psi4.py#L196).


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#402 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AC9QdgSg1IZHU-ezlwNRXjWKX3oEMSQaks5q-2JfgaJpZM4IvHrd
.

@susilehtola

This comment has been minimized.

Show comment
Hide comment
@susilehtola

susilehtola Feb 10, 2017

Contributor

detci memory size checks would be nice. Signed, someone who has crashed his computer a few times by trying to do a too big calculation.

Contributor

susilehtola commented Feb 10, 2017

detci memory size checks would be nice. Signed, someone who has crashed his computer a few times by trying to do a too big calculation.

@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Mar 7, 2017

Member
  • Implement logging for PsiAPI mode so that output files (currently w/o Psi4 header or any record of input commands or options) aren't so mysterious. At minimum, print to output the psi4.header() at file creation and e.g. >>> aggh = psi4.energy("scf") and >>> psi4.set_options({"scf_type": "pk"}) to track what was done.
Member

loriab commented Mar 7, 2017

  • Implement logging for PsiAPI mode so that output files (currently w/o Psi4 header or any record of input commands or options) aren't so mysterious. At minimum, print to output the psi4.header() at file creation and e.g. >>> aggh = psi4.energy("scf") and >>> psi4.set_options({"scf_type": "pk"}) to track what was done.
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Mar 15, 2017

Member
  • API-like loading of basis sets. If one has the coef and the exp, shouldn't have to write them out to a .gbs, should be a constructor to take them directly.
Member

loriab commented Mar 15, 2017

  • API-like loading of basis sets. If one has the coef and the exp, shouldn't have to write them out to a .gbs, should be a constructor to take them directly.
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab May 3, 2017

Member
Member

loriab commented May 3, 2017

@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab May 15, 2017

Member
  • Matt Schieber has info on KMP affinity and thread binding schemes that hurt and help performance. See if the one that helps can be encoded at runtime (knowing some /proc/cpuinfo) or otherwise advise in docs.
Member

loriab commented May 15, 2017

  • Matt Schieber has info on KMP affinity and thread binding schemes that hurt and help performance. See if the one that helps can be encoded at runtime (knowing some /proc/cpuinfo) or otherwise advise in docs.
@susilehtola

This comment has been minimized.

Show comment
Hide comment
@susilehtola

susilehtola May 18, 2017

Contributor
  • CISD module should be able to compute density matrices, so that the NOs could be used as initial guesses for detci
Contributor

susilehtola commented May 18, 2017

  • CISD module should be able to compute density matrices, so that the NOs could be used as initial guesses for detci
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab May 24, 2017

Member
  • [ ]
sp = single_point('scf')
sp.compute() # ?
wfn = sp.wfn()
energy = sp.energy()
Member

loriab commented May 24, 2017

  • [ ]
sp = single_point('scf')
sp.compute() # ?
wfn = sp.wfn()
energy = sp.energy()
@susilehtola

This comment has been minimized.

Show comment
Hide comment
@susilehtola

susilehtola May 25, 2017

Contributor
  • Implement O2 method, i.e. orbital-optimized scaled opposite-spin MP2 [R. C. Lochan and M. Head-Gordon, J. Chem. Phys. 126, 164101 (2007)]. (Looks like there is a scaled opposite-spin MP2 method in dfocc?)
Contributor

susilehtola commented May 25, 2017

  • Implement O2 method, i.e. orbital-optimized scaled opposite-spin MP2 [R. C. Lochan and M. Head-Gordon, J. Chem. Phys. 126, 164101 (2007)]. (Looks like there is a scaled opposite-spin MP2 method in dfocc?)
@susilehtola

This comment has been minimized.

Show comment
Hide comment
@susilehtola

susilehtola May 31, 2017

Contributor
  • Implement exchange guess for virtual orbitals for CAS calculations. (Improved virtual orbitals by rediagonalizing virtual space with respect to exchange operator.)
Contributor

susilehtola commented May 31, 2017

  • Implement exchange guess for virtual orbitals for CAS calculations. (Improved virtual orbitals by rediagonalizing virtual space with respect to exchange operator.)
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Sep 20, 2017

Member
  • Unify origin-setting for properties/integrals. Properties by oeprop get their origin from options (https://github.com/psi4/psi4/blob/master/psi4/src/psi4/libmints/oeprop.cc#L775-L808). Mints integrals get their origin from default 0,0,0 or from options. But with Mints, you sometimes want to set the origin by argument (default 0,0,0) in API. Unification so that useful from both input file and API would be nice. Also maybe an easy center-of-charge spec.
Member

loriab commented Sep 20, 2017

  • Unify origin-setting for properties/integrals. Properties by oeprop get their origin from options (https://github.com/psi4/psi4/blob/master/psi4/src/psi4/libmints/oeprop.cc#L775-L808). Mints integrals get their origin from default 0,0,0 or from options. But with Mints, you sometimes want to set the origin by argument (default 0,0,0) in API. Unification so that useful from both input file and API would be nice. Also maybe an easy center-of-charge spec.
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Oct 5, 2017

Member
  • Can libmints get away with not having MAX_AM_ERI passed to it? That way one could switch out libint.sos on the fly (conda can do this). Would have to find a way to still exit gracefully if insufficient AM and give decent error messages.
Member

loriab commented Oct 5, 2017

  • Can libmints get away with not having MAX_AM_ERI passed to it? That way one could switch out libint.sos on the fly (conda can do this). Would have to find a way to still exit gracefully if insufficient AM and give decent error messages.
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Nov 4, 2017

Member
  • Examine properties function in driver (that encodes the extended workflows beyond sp or time-trivial keyword needed to compute CC properties) and the oeprop syntax that most all other methods use. Devise and implement a unified interface that still works with the code.
Member

loriab commented Nov 4, 2017

  • Examine properties function in driver (that encodes the extended workflows beyond sp or time-trivial keyword needed to compute CC properties) and the oeprop syntax that most all other methods use. Devise and implement a unified interface that still works with the code.
@robertodr

This comment has been minimized.

Show comment
Hide comment
@robertodr

robertodr Nov 4, 2017

Contributor
  • Add FCIDUMP capabilities within Psi4. There are currently many third-party plugins that do that and it seems needed enough to have it within the main code.

Done in #872

Contributor

robertodr commented Nov 4, 2017

  • Add FCIDUMP capabilities within Psi4. There are currently many third-party plugins that do that and it seems needed enough to have it within the main code.

Done in #872

@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Nov 8, 2017

Member

Beginner-Friendly (after py-side SCF)

Renovate SCF damping capabilities

  • right now we're only damping D. There are rumors that should also damp F or there's an iteration mismatch between D & F. Make sure damping is working right (presently only test case is scf-coverage and damping severely hurts convergence
  • right now we damp to the very end of iterations. Maybe damp only to 1e-4 (after which soscf might be better choice, now that DGAS has implemented) controlled by DAMPING_CONVERGENCE.
  • here's a paper with potentially useful schemes https://arxiv.org/pdf/1302.6099.pdf
  • consider (this may be separate project) more advanced driver logic. perhaps something along the lines of (DGAS) "Damp first iteration, then DIIS, if density is stuck take stability step, if grad < 1.e-4 and target < 1.e-8 take SOSCF, else DIIS. if density oscillation (dRMS/oscilation) > 1.e2 add damping"
Member

loriab commented Nov 8, 2017

Beginner-Friendly (after py-side SCF)

Renovate SCF damping capabilities

  • right now we're only damping D. There are rumors that should also damp F or there's an iteration mismatch between D & F. Make sure damping is working right (presently only test case is scf-coverage and damping severely hurts convergence
  • right now we damp to the very end of iterations. Maybe damp only to 1e-4 (after which soscf might be better choice, now that DGAS has implemented) controlled by DAMPING_CONVERGENCE.
  • here's a paper with potentially useful schemes https://arxiv.org/pdf/1302.6099.pdf
  • consider (this may be separate project) more advanced driver logic. perhaps something along the lines of (DGAS) "Damp first iteration, then DIIS, if density is stuck take stability step, if grad < 1.e-4 and target < 1.e-8 take SOSCF, else DIIS. if density oscillation (dRMS/oscilation) > 1.e2 add damping"
@susilehtola

This comment has been minimized.

Show comment
Hide comment
@susilehtola

susilehtola Nov 8, 2017

Contributor

Related to @loriab 's previous comment... damping F is a lot more better than damping D, since instead of intrapolating inside a fixed Hilbert space of density, you're actually looking at different densities. You converge faster if you damp F than if you just damp D, and you're less likely to land on a saddle point.

Contributor

susilehtola commented Nov 8, 2017

Related to @loriab 's previous comment... damping F is a lot more better than damping D, since instead of intrapolating inside a fixed Hilbert space of density, you're actually looking at different densities. You converge faster if you damp F than if you just damp D, and you're less likely to land on a saddle point.

@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Nov 14, 2017

Member

Make use of doublet/triplet c-side. In particular, https://github.com/psi4/psi4/pull/737/files#r121282237

Member

loriab commented Nov 14, 2017

Make use of doublet/triplet c-side. In particular, https://github.com/psi4/psi4/pull/737/files#r121282237

@loriab loriab referenced this issue Nov 27, 2017

Closed

reorganize Hessian, vib, thermo analysis #347

0 of 12 tasks complete
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Feb 2, 2018

Member

things we can do after dumping py27 (just to keep track)

  • delete all __future__ statements
  • simplify molutil and qcdb/molecule.py raw_* functions
Member

loriab commented Feb 2, 2018

things we can do after dumping py27 (just to keep track)

  • delete all __future__ statements
  • simplify molutil and qcdb/molecule.py raw_* functions
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Mar 6, 2018

Member

[from @dgasmith psi4/psi4#928 ] File names are going haywire a bit with different conventions for either PSIO or "user facing names" like Molden and cube files. Anyone have a good idea how to rectify this? It would also be good to have restart work for gradients, Hessians, and properties as well. Ill add this to the wish ilst.

Member

loriab commented Mar 6, 2018

[from @dgasmith psi4/psi4#928 ] File names are going haywire a bit with different conventions for either PSIO or "user facing names" like Molden and cube files. Anyone have a good idea how to rectify this? It would also be good to have restart work for gradients, Hessians, and properties as well. Ill add this to the wish ilst.

@j3mdamas

This comment has been minimized.

Show comment
Hide comment
@j3mdamas

j3mdamas Mar 7, 2018

As per discussed with @loriab in #933, it would be interesting to have a Windows native compilation for psi4

j3mdamas commented Mar 7, 2018

As per discussed with @loriab in #933, it would be interesting to have a Windows native compilation for psi4

@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Apr 29, 2018

Member

Possible easy-start issues

Member

loriab commented Apr 29, 2018

Possible easy-start issues

@robertodr

This comment has been minimized.

Show comment
Hide comment
@robertodr

robertodr May 2, 2018

Contributor
  • PCM gradients à la ECP, whereby the in vacuo terms are computed analytically and the PCM terms numerically. This can actually be done already on top of #953.
  • PCM analytic gradients. This is more of a PCMSolver-side problem and a time investment I can't currently make. For those interested, there is this (very broken) PCMSolver/pcmsolver#140 which should make much easier implementing the cavity terms for the gradient and Hessian.
Contributor

robertodr commented May 2, 2018

  • PCM gradients à la ECP, whereby the in vacuo terms are computed analytically and the PCM terms numerically. This can actually be done already on top of #953.
  • PCM analytic gradients. This is more of a PCMSolver-side problem and a time investment I can't currently make. For those interested, there is this (very broken) PCMSolver/pcmsolver#140 which should make much easier implementing the cavity terms for the gradient and Hessian.
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Jun 15, 2018

Member

Plugin wish list:

  • allow compiled pluginname.so to have a name other than matching import pluginname, so that python import system can't mistake the .so for the actual module. (see edeprince3/gpu_dfcc#2 (comment))
Member

loriab commented Jun 15, 2018

Plugin wish list:

  • allow compiled pluginname.so to have a name other than matching import pluginname, so that python import system can't mistake the .so for the actual module. (see edeprince3/gpu_dfcc#2 (comment))
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Jun 15, 2018

Member

From @dgasmith, relevant to the smart SCF driver

  • We should look into a SCF mode that saves some grid components, they are not that big and the savings can add up for high iterations.
Member

loriab commented Jun 15, 2018

From @dgasmith, relevant to the smart SCF driver

  • We should look into a SCF mode that saves some grid components, they are not that big and the savings can add up for high iterations.
@loriab

This comment has been minimized.

Show comment
Hide comment
@loriab

loriab Jul 17, 2018

Member

So we don't forget it

Member

loriab commented Jul 17, 2018

So we don't forget it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment