SCAN/libXC segmentation fault

[hokru]

This was already shortly discussed on the forum http://forum.psicode.org/t/libxc-density-functionals/589, possibly caused by libxc. No idea if this can be hot-fixed without upgrade to libxc4, but SCAN-D3(BJ) gave excellent results for the GMTKN55 so it would be really nice to have it.

Here is a test than runs scan and scan0: scan.txt

also tagging @dgasmith because he did already some checks on this.

[dgasmith]

After chatting with the LibXC dev's I think we can do a few workarounds so that LibXC 4 works for us. If I get a second I might work on getting that upgrade through.

[susilehtola]

Hm, any reason why Psi4 isn't asking libxc for the default exact exchange parameters?

[susilehtola]

And yeah, scan correlation crashes because it's not implemented in libxc 3.0.0...

[dgasmith]

@susilehtola Then why do the SCAN primitives exist in 3.0? See here for a 3.0 clone.

[susilehtola]

@dgasmith .. because they went ahead and published 3.0.0 before scan correlation had been filled in. See
https://gitlab.com/libxc/libxc/blob/3.0.0/src/mgga_c_scan.c#L314

[dgasmith]

@susilehtola Ah, I see thank you. Is there a way to detect this in LibXC 4.0?

[susilehtola]

Miguel implemented it after 3.0.0, but after that we switched over to machine generated code. Libxc 4 is probably better for you anyway.

[susilehtola]

Detect what? Libxc 4 has SCAN.

[dgasmith]

Detect non-operational functionals ahead of time.

[susilehtola]

Not really. Libxc4 doesn't have anything that crashes.

I've backported the SCAN implementation to libxc 3 at
https://gitlab.com/libxc/libxc/commit/96568b1dd36130df57d19f7037fab7afecde48a4
as it appears there's going to be a new release in that series soon.

[susilehtola]

libxc 3.0.1 has now been released, which includes my backport of SCAN correlation.

Still, I would recommend migrating to libxc 4, where the functional implementations are autogenerated using Maple.

[hokru]

@susilehtola good news, thank you!

I downloaded the 3.0.1 tarball, compiled with -fPIC and replaced the libxc.a and header files in my psi4 installation with the new ones. Then re-linked psi4. Seemed the quickest way, but not 100% sure this is fully correct.

Instead of a segmentation fault I get nans

  ==> Iterations <==

                           Total Energy        Delta E     RMS |[F,P]|

   @DF-RKS iter   0:                 -nan           -nan   -nan        

edit: replacing C_SCAN with C_TPSS works gives normal results.

[susilehtola]

Yes, this is the second issue that I mentioned on the forum. SCAN is a ridiculously ill conditioned functional. It needs hundreds of radial grid points to converge to the grid limit, and also appears to produce NaNs. The latter issue also appears in libxc 4..

[hokru]

odd, I know of the increased grid dependency from colleagues. But both qchem and Grimme's manual SCAN implementation in Turbomole do not seem to have NaN issues.

[dgasmith]

We could add a problematic_functional flag where we work a bit on the input/output. Should we scan out densities in the input, NaN's in the output, or both?

[loriab]

@hokru, psi won't be able to update to 3.0.1 in general because the cmake buildsys wasn't backported. But so long as you can get a v3 libxc.a or .so out of libtools and stuff things in the right place, the "hand update" you describe should be perfectly legit.

[susilehtola]

@dgasmith do you not have density thresholding in the dft code? It appears to be a common trick to make dft integration faster to avoid calculating zeros...

I think NaNs should be sieved out in the output.

[susilehtola]

Yes, the NaN thing is weird, especially IIRC it appears both in the old manual implementation, and the new automatically generated implementation. Maybe we'll be able to dig out the problem...

[susilehtola]

@loriab aren't you still using your own libxc repo? You can just update those files to 3.0.1...?

[dgasmith]

@susilehtola We usually screen out small values via localization of the collocation grid. We don't really check the density itself, shouldn't be hard though. I can look into it a bit to see if its worth it.

[loriab]

True. But this changeset is hefty enough that I'd rather not sort and hand-apply it when (1) Holger has a functional work-around and (2) Libxc4 is the real goal.

[dgasmith]

@susilehtola Actually after thinking about it, shouldn't LibXC do the nan scan and filter out these values? The user should filter through the input set for speed, but the downstream program probably shouldn't be responsible for output massaging as this will lead to everyone using LibXC fiddling with this (in probably different manners) when it should be fixed in a single place.

[susilehtola]

@dgasmith .... of course it should, silly me.

[dgasmith]

Tossing this one back to LibXC and closing.