Adds r2SCAN hybrids; r2SCAN-3c and B97-3c #2842

hokru · 2023-01-04T11:43:22Z

Description

Adds new density functional approximations and Grimme group composite methods.

closes #2121 #1898

User API & Changelog headlines

new DFA: r2SCAN0, r2SCANh, r2SCAN50 (plus -D4 versions)
new DFA composite methods: r2SCAN-3C, B97-3C
new basis sets for the composite methods: def2-mTZVP, def2-mTZVPP

Dev notes & details

B97-3c requires s-dftd3 and mctc-gcp
r2SCAN-3c requires mctc-gcp and dftd4 3.5.0
[dispersion]['params'] functional dictionary can be incomplete. Defaults will be added automatically.
allow parameter tweaking of pre-defined LibXC functionals.
keep compatibility with LibXC 5 for windows
requires qcengine update update gCP levels and extend D4 defaults MolSSI/QCEngine#393 (review)
and dftd4-python with Allow passing through values for dispersion model dftd4/dftd4#184
needs Libxc v6.0
adds wb97x-3c closes Implement wB97X-3c #2850

Checklist

Tests added for any new features
All or relevant fraction of full tests run

Status

Ready for review
Ready for merge

susilehtola · 2023-01-12T17:55:09Z

Where are the basis sets from? Ideally they should just go straight to https://github.com/MolSSI-BSE/basis_set_exchange

The r2SCAN hybrids are available in the current version of Libxc.

hokru · 2023-01-12T20:06:34Z

Where are the basis sets from? Ideally they should just go straight to https://github.com/MolSSI-BSE/basis_set_exchange

I took the basis from ORCA (orca_exportbasis tool), converted with bse conversion tool and appended our existing def2-ECPs.

loriab · 2023-02-24T21:24:20Z

Hi @hokru, I uploaded psi4/label/dev::dftd4-3.5.0 packages (py intf present, so these are equiv to conda-forge::dftd4-python and tried out this branch with a mix of packages so that I could add some more helpful error messages. So there's one commit here and two over at qcengine. Also I hopefully got around all the CI checks here, but we'll see.

Locally, this passes with c-f::gcp-correction and psi4::dftd4

# Any line starting with the # character is a comment line
#! Sample HF/cc-pVDZ H2O computation

memory 600 mb

molecule h2o {
  O 
  H 1 0.96
  H 1 0.96 2 104.5
}

set basis cc-pVDZ
ene = energy('r2scan-d4')
compare_values(-76.38149185, ene, 6, 'asdf')

energy('r2scan-3c/')  # slash unsets basis from above
compare_values(-76.41834614, variable('SCF TOTAL ENERGY'), 6, 'SCF energy')  #TEST

loriab · 2023-02-25T04:12:48Z

@hokru I get the error below when there's no dftd4 of any kind around. (It's what Azure linux is doing now.) When I change the try/except in qcng empirical_disp_param line 906 from ModuleNotFoundError to ImportError. This is py310, so I'm not sure why, but it might be a sol'n.

# packages in environment at /psi/toolchainconda/envs/py310:
#
# Name                    Version                   Build  Channel
dftd3                     3.2.1                h84218bc_2    psi4
(py310) psilocaluser@bash:psinet:/psi/gits/hrw-detangle/objdir_py310: (r2scan) conda list gcp
# packages in environment at /psi/toolchainconda/envs/py310:
#
# Name                    Version                   Build  Channel
gcp                       2.0.2                he991be0_2    psi4
gcp-correction            2.3.1                h1990efc_3    conda-forge
(py310) psilocaluser@bash:psinet:/psi/gits/hrw-detangle/objdir_py310: (r2scan) pytest ../tests/pytests/test_psi4.py 
================================================================================== test session starts ==================================================================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /psi/gits/hrw-detangle, configfile: pytest.ini
plugins: forked-1.3.0, xdist-2.5.0
collected 0 items / 1 error                                                                                                                                                             

======================================================================================== ERRORS =========================================================================================
______________________________________________________________________ ERROR collecting tests/pytests/test_psi4.py ______________________________________________________________________
ImportError while importing test module '/psi/gits/hrw-detangle/tests/pytests/test_psi4.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../tests/pytests/test_psi4.py:4: in <module>
    import psi4
stage/lib/psi4/__init__.py:75: in <module>
    from .driver import endorsed_plugins
stage/lib/psi4/driver/__init__.py:46: in <module>
    from psi4.driver.molutil import *
stage/lib/psi4/driver/molutil.py:36: in <module>
    from psi4.driver.p4util import temp_circular_import_blocker
stage/lib/psi4/driver/p4util/__init__.py:35: in <module>
    from .fcidump import *
stage/lib/psi4/driver/p4util/fcidump.py:43: in <module>
    from psi4.driver.procrouting.proc_util import check_iwl_file_from_scf_type
stage/lib/psi4/driver/procrouting/__init__.py:29: in <module>
    from .proc_table import procedures, hooks, energy_only_methods, integrated_basis_methods
stage/lib/psi4/driver/procrouting/proc_table.py:34: in <module>
    from . import sapt
stage/lib/psi4/driver/procrouting/sapt/__init__.py:29: in <module>
    from .sapt_proc import run_sapt_dft, sapt_dft, run_sf_sapt
stage/lib/psi4/driver/procrouting/sapt/sapt_proc.py:36: in <module>
    from psi4.driver.procrouting.proc import scf_helper
stage/lib/psi4/driver/procrouting/proc.py:58: in <module>
    from . import proc_util
stage/lib/psi4/driver/procrouting/proc_util.py:37: in <module>
    from psi4.driver.procrouting.dft import functionals, build_superfunctional_from_dictionary
stage/lib/psi4/driver/procrouting/dft/__init__.py:29: in <module>
    from .superfunctionals import build_superfunctional
stage/lib/psi4/driver/procrouting/dft/superfunctionals.py:37: in <module>
    from . import dft_builder
stage/lib/psi4/driver/procrouting/dft/dft_builder.py:86: in <module>
    from qcengine.programs.empirical_dispersion_resources import dashcoeff, get_dispersion_aliases, new_d4_api
stage/lib/qcengine/__init__.py:6: in <module>
    from .compute import compute, compute_procedure
stage/lib/qcengine/compute.py:11: in <module>
    from .procedures import get_procedure
stage/lib/qcengine/procedures/__init__.py:1: in <module>
    from .base import get_procedure, list_all_procedures, list_available_procedures, register_procedure
stage/lib/qcengine/procedures/base.py:10: in <module>
    from .nwchem_opt import NWChemDriverProcedure
stage/lib/qcengine/procedures/nwchem_opt/__init__.py:7: in <module>
    from qcengine.procedures.nwchem_opt.harvester import harvest_as_atomic_result
stage/lib/qcengine/procedures/nwchem_opt/harvester.py:8: in <module>
    from qcengine.programs.nwchem.harvester import harvest_outfile_pass
stage/lib/qcengine/programs/__init__.py:1: in <module>
    from .base import get_program, list_all_programs, list_available_programs, register_program, unregister_program
stage/lib/qcengine/programs/base.py:11: in <module>
    from .dftd3 import DFTD3Harness
stage/lib/qcengine/programs/dftd3.py:19: in <module>
    from . import empirical_dispersion_resources
stage/lib/qcengine/programs/empirical_dispersion_resources.py:906: in <module>
    from dftd4 import __version__ as d4_version
E   ImportError: cannot import name '__version__' from 'dftd4' (unknown location)
================================================================================ short test summary info ================================================================================

loriab · 2023-08-10T08:12:33Z

I've tweaked the https://github.com/MolSSI/QCEngine/pull/393/files PR slightly, and I think it's ready for merge. Here, I've added a few tests (b97-3c, r2scan2-3c, and r2scan2-d4) and tried them with several combinations of old/new gcp, old/new dftd3, and 3.4/3.5 dftd4, so I'm semi-confident they run when they ought and give helpful errors when they can't. The tests I added cover energy and gradient, but they're purely internal -- no reference values from a separate implementation. What further tests are wanted?

The labels/selectors for pytest should be in good shape (that is, tests will skip if wrong gcp/d3/d4 detected). CTest selectors aren't in good shape, so if you get helpful "can't run" errors, that's expected; wrong values aren't expected.

loriab · 2023-08-10T15:28:59Z

external/upstream/qcengine/CMakeLists.txt

+        #URL https://github.com/MolSSI/QCEngine/archive/v0.28.0.tar.gz
+        #URL https://github.com/hokru/QCEngine/archive/3cupdate.tar.gz
+        URL https://github.com/loriab/QCEngine/archive/v0.28.0.dev2.tar.gz  # == hokru:3cupdate but tag needed for pip


Suggested change

#URL https://github.com/MolSSI/QCEngine/archive/v0.28.0.tar.gz

#URL https://github.com/hokru/QCEngine/archive/3cupdate.tar.gz

URL https://github.com/loriab/QCEngine/archive/v0.28.0.dev2.tar.gz # == hokru:3cupdate but tag needed for pip

#URL https://github.com/MolSSI/QCEngine/archive/v0.28.0.tar.gz

URL https://github.com/MolSSI/QCEngine/archive/v0.28.0.dev3.tar.gz # includes hokru:3cupdate; tag needed for pip

when resuming work on this PR, switch to this tag if v0.28 not yet minted. I don't want to trigger the full CI or I'd push it.

susilehtola · 2023-08-10T19:16:19Z

The basis sets were merged to BSE in

MolSSI-BSE/basis_set_exchange#276

but some issues are still fixed in

MolSSI-BSE/basis_set_exchange#282

One should use the basis sets from the BSE.

susilehtola · 2023-08-10T19:17:17Z

psi4/driver/procrouting/dft/hyb_functionals.py

+    "name": "R2SCAN0",
+    "x_functionals": {
+        "MGGA_X_R2SCAN": {"alpha": 0.75}
+    },
+    "c_functionals": {
+        "MGGA_C_R2SCAN": {}
+    },
+    "x_hf": {
+        "alpha": 0.25
+    },
+    "description": '    r2SCAN0 Hybrid Meta-GGA XC Functional\n',
+    "citation": '    M. Bursch, H. Neugebauer, S. Ehlert, S. Grimme  J. Chem. Phys. 156, 134105, 2022   \n',
+    "doi": "10.1063/5.0086040",


Just use XC_HYB_MGGA_XC_R2SCAN0

Right now we're supporting libxc v.5.1.2 ... v6.x . Would that range have to be curtailed to use the all-in-one fctl definitions?

iirc, we ended up supporting two major versions because v6 came out right before a psi4 release, and we didn't want to demand a sudden change. That is, there's no harm in using only v6 now that c-f can build new Windows versions for us.

Yeah the r2scan hybrids were added in 6.1.0.

ok, let's plan on dropping v5 and requiring minimum v6.1 then.

susilehtola · 2023-08-10T19:17:30Z

psi4/driver/procrouting/dft/hyb_functionals.py

+    "name": "R2SCANh",
+    "x_functionals": {
+        "MGGA_X_R2SCAN": {"alpha": 0.90}
+    },
+    "c_functionals": {
+        "MGGA_C_R2SCAN": {}
+    },
+    "x_hf": {
+        "alpha": 0.10
+    },
+    "description": '    r2SCANh Hybrid Meta-GGA XC Functional\n',
+    "citation": '    M. Bursch, H. Neugebauer, S. Ehlert, S. Grimme  J. Chem. Phys. 156, 134105, 2022   \n',
+    "doi": "10.1063/5.0086040",


Just use XC_HYB_MGGA_XC_R2SCANH

susilehtola · 2023-08-10T19:17:44Z

psi4/driver/procrouting/dft/hyb_functionals.py

+    "name": "R2SCAN50",
+    "x_functionals": {
+        "MGGA_X_R2SCAN": {"alpha": 0.50}
+    },
+    "c_functionals": {
+        "MGGA_C_R2SCAN": {}
+    },
+    "x_hf": {
+        "alpha": 0.50
+    },
+    "description": '    r2SCAN50 Hybrid Meta-GGA XC Functional\n',
+    "citation": '    M. Bursch, H. Neugebauer, S. Ehlert, S. Grimme  J. Chem. Phys. 156, 134105, 2022   \n',


Just use XC_HYB_MGGA_XC_R2SCAN50

aizvorski · 2023-09-24T01:35:39Z

Hi folks - I'd like to help with this, although I'm not familiar with the psi4 code internals. Do you need reference values for the tests? (on what systems?)

I'm wondering if the gcp part of r2SCAN-3c would affect the results in any test with a single molecule; do we need something like a water dimer?

loriab · 2023-09-24T05:57:27Z

Thanks for your interest and the offer, @aizvorski. Looking back over this, the necessary QCEngine has now been released, so it looks like my first steps are updating that CI and limiting Libxc to v6 so the functional definitions can be simplified. Certainly intermolecular tests are desirable. There's some ref values here http://www.thch.uni-bonn.de/tc.old/downloads/GMTKN/GMTKN55/functional/r2SCAN-3c.html but largely they need hunting down in either papers or an independent implementation. I like S22 but anything independent will do. Then Susi had some basis set comments that I haven't looked into yet.

loriab · 2023-11-28T17:52:24Z

@aizvorski or @hokru (or anyone), did you happen to have on hand independent ref values for any of these five fctls, particularly r2SCAN-3C & B97-3C ? (Note that c-f dftd4 isn't available for py312, so I'll likely need to adjust the CI to improve testing coverage. If this makes it into v1.9, I'm good with dropping support for dftd3 classic and the psi4 build of dftd4. Also, the plan is libxc=6, dropping v5 support.)

loriab · 2023-12-03T10:47:47Z

This adds external comparisons for the r2SCAN hybrids, r2scan-3c, B97-3c. It also implements wB97X-3c. I think it's good to go. (well, CI/test-skipping may yet need tweaks.) But @susilehtola or @hokru might want to look it over.

susilehtola · 2023-12-04T08:41:39Z

psi4/driver/procrouting/dft/gga_functionals.py

+        # "GGA_XC_B97_3C": {} #for libxc >= v6.0
+        "GGA_XC_B97_D": {
+            "tweak": { # needed until libxc >= v6.0
+            "_cx0": 1.076616,
+            "_cx1":-0.469912,
+            "_cx2":3.322442,
+            "_css0":0.543788,
+            "_css1":-1.444420,
+            "_css2":1.637436,
+            "_cos0":0.635047,
+            "_cos1":5.532103,
+            "_cos2":-15.301575,
+            },


Can be dropped, no?

davpoolechem · 2023-12-04T18:54:16Z

doc/sphinxman/source/dftd3.rst

@@ -222,6 +223,7 @@ available only through the ``DFTD3`` or ``DFTD4`` programs. Once installed, the
 ``dftd3``/|PSIfour| and ``dftd4``/|PSIfour| interfaces are transparent, and all corrections are
 interfaced exactly alike.
 The -D3 interface can use classic or simple-dftd3 programs interchangeably and will prefer the latter.
+Starting in v1.9, the classic program is no longer supported or tested, though it won't be deliberately disabled.


Suggested change

Starting in v1.9, the classic program is no longer supported or tested, though it won't be deliberately disabled.

Starting in v1.9, the classic program is no longer supported or tested, though it isn't deliberately disabled.

Slightly more grammatically consistent with my suggestion imo, but feel free to ignore if you don't agree.

Probably better since I don't want to promise to never disable the classic. I'll add it to a different PR if that's the only change, since CI is already complete here.

davpoolechem · 2023-12-04T20:22:46Z

psi4/src/psi4/libfunctional/superfunctional.cc

@@ -99,6 +99,11 @@ std::shared_ptr<SuperFunctional> SuperFunctional::XC_build(std::string name, boo
    sup->set_x_alpha(xc_func->global_exchange());
    sup->set_x_beta(xc_func->lr_exchange());

+    // User tweakers
+    if (tweakers_->size()) {


Should there be a check here for the case that tweakers_ is uninitialized, i.e., std::nullopt?

The actual use of the tweakers for this PR is all removed by now by requiring libxc v6, but it's good to future proof. What do you suggest?

Ah, got it! I suppose that's a new feature of the code I missed. But regarding suggestions, something like:

if (tweakers_.value().size()) {

Will cause the code to throw an exception if tweakers_ is uninitialized, acting as the check. If you want to avoid throwing entirely, you could instead enclose this if statement in another if-block:

if (tweakers_.has_value()) { ... }

to ensure that tweakers_ must be initialized for the set_tweak call to proceed.

Ok, fixed. One doesn't want the has_value version b/c that always tries to set tweak and some/most? libxc fctls aren't tweakable.

hokru mentioned this pull request Jan 5, 2023

update gCP levels and extend D4 defaults MolSSI/QCEngine#393

Merged

2 tasks

hokru marked this pull request as ready for review January 5, 2023 13:05

hokru mentioned this pull request Jan 12, 2023

Implement wB97X-3c #2850

Closed

loriab added this to the Psi4 1.8 milestone Feb 25, 2023

loriab added enhancement external-interface For issues about interfaces with external programs: ADCC, CheMPS2, GDMA, MRCC... dft For issues specific to DFT and their many functionals. labels Feb 25, 2023

loriab modified the milestones: Psi4 1.8, Psi4 1.9 May 8, 2023

loriab force-pushed the r2scan branch from 252e6d2 to 3184314 Compare August 10, 2023 05:18

loriab reviewed Aug 10, 2023

View reviewed changes

susilehtola reviewed Aug 10, 2023

View reviewed changes

hokru added 8 commits November 28, 2023 02:09

adds couple of r2scan based functionals

63050a0

3c update, + scan hybrid functionals

041f5a8

dft printing doi formatting

8e36e00

allow omitting default disp params; turn gcp on

4467a43

cleanup

b467e22

adds basis set files

666c14a

add <optional> header

38b2525

prevent r2scan3c dict from building; get new QCEngine

ac8014b

loriab added 11 commits November 28, 2023 02:17

allow custom qcng in CI

c298522

py packaging wants to control us

c7b8766

right number

01589f6

ok, setuptools via packaging wants to control us

2e13c24

they're working together

c307739

one back

6191bf9

more errors

3a4205f

fix bib

754e03e

full spec

a1abd8b

add some tests

c483d7f

touchups

32ab5d3

loriab force-pushed the r2scan branch from dd2930a to 32ab5d3 Compare November 28, 2023 07:34

check everything

931069d

loriab added 2 commits December 4, 2023 01:14

fixup

2364a74

fixups

9c73f42

susilehtola reviewed Dec 4, 2023

View reviewed changes

susilehtola approved these changes Dec 4, 2023

View reviewed changes

condense another fctl

07bae78

davpoolechem reviewed Dec 4, 2023

View reviewed changes

fixes

7eac4f0

davpoolechem approved these changes Dec 5, 2023

View reviewed changes

loriab enabled auto-merge December 5, 2023 01:24

loriab added this pull request to the merge queue Dec 5, 2023

Merged via the queue into psi4:master with commit 6c81789 Dec 5, 2023

loriab mentioned this pull request Dec 5, 2023

implement B97-3c composite method #1898

Closed

5 tasks

loriab mentioned this pull request Jan 12, 2024

WIP: Implement def2-mTZVP and def2-mTZVPP MolSSI-BSE/basis_set_exchange#197

Open

	Starting in v1.9, the classic program is no longer supported or tested, though it won't be deliberately disabled.
	Starting in v1.9, the classic program is no longer supported or tested, though it isn't deliberately disabled.

Adds r2SCAN hybrids; r2SCAN-3c and B97-3c #2842

Adds r2SCAN hybrids; r2SCAN-3c and B97-3c #2842

Uh oh!

Conversation

hokru commented Jan 4, 2023 • edited by loriab Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

User API & Changelog headlines

Dev notes & details

Checklist

Status

Uh oh!

susilehtola commented Jan 12, 2023

Uh oh!

hokru commented Jan 12, 2023

Uh oh!

loriab commented Feb 24, 2023

Uh oh!

loriab commented Feb 25, 2023

Uh oh!

loriab commented Aug 10, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

susilehtola commented Aug 10, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aizvorski commented Sep 24, 2023

Uh oh!

loriab commented Sep 24, 2023

Uh oh!

loriab commented Nov 28, 2023

Uh oh!

loriab commented Dec 3, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davpoolechem Dec 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hokru commented Jan 4, 2023 •

edited by loriab

Loading

davpoolechem Dec 4, 2023 •

edited

Loading