Hbond excl fix #3242

richardjgowers · 2021-04-25T16:34:59Z

Fixes #2987

Changes made in this Pull Request:

PR Checklist

Tests?
Docs?
CHANGELOG updated?
Issue raised/referenced?

pep8speaks · 2021-04-25T16:35:03Z

Hello @richardjgowers! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py:

Line 328:80: E501 line too long (93 > 79 characters)
Line 433:80: E501 line too long (99 > 79 characters)

In the file testsuite/MDAnalysisTests/lib/test_cutil.py:

Line 87:63: W605 invalid escape sequence '('
Line 87:69: W605 invalid escape sequence ')'
Line 87:80: E501 line too long (80 > 79 characters)

Comment last updated at 2021-04-28 07:32:55 UTC

codecov · 2021-04-25T16:50:23Z

Codecov Report

Merging #3242 (50531d9) into master (361ca7e) will increase coverage by 0.84%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3242      +/-   ##
==========================================
+ Coverage   90.99%   91.84%   +0.84%     
==========================================
  Files         162      167       +5     
  Lines       22217    22737     +520     
  Branches     3203     3203              
==========================================
+ Hits        20216    20882     +666     
- Misses       1378     1771     +393     
+ Partials      623       84     -539

Impacted Files	Coverage Δ
package/MDAnalysis/lib/distances.py	`98.42% <ø> (+0.94%)`	⬆️
...age/MDAnalysis/analysis/hbonds/hbond_autocorrel.py	`95.00% <100.00%> (+7.10%)`	⬆️
package/MDAnalysis/lib/_cutil.pyx	`100.00% <100.00%> (ø)`
package/MDAnalysis/topology/tpr/obj.py	`96.82% <0.00%> (-3.18%)`	⬇️
...onality_reduction/DimensionalityReductionMethod.py	`97.14% <0.00%> (-2.86%)`	⬇️
package/MDAnalysis/due.py	`56.09% <0.00%> (ø)`
package/MDAnalysis/analysis/legacy/x3dna.py	`0.00% <0.00%> (ø)`
package/MDAnalysis/tests/datafiles.py	`31.25% <0.00%> (ø)`
package/MDAnalysis/tests/__init__.py	`100.00% <0.00%> (ø)`
package/MDAnalysis/analysis/legacy/__init__.py	`0.00% <0.00%> (ø)`
... and 88 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 361ca7e...50531d9. Read the comment docs.

IAlibay

Couple of tests & logic comments.

Also my cython is probably too green to probably review stuff, if someone with more experience can have a look that'd be great - pinging @MDAnalysis/coredevs

package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py

IAlibay · 2021-04-25T17:00:10Z

package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py

+        pair = capped_distance(self.h.positions, self.a.positions, max_cutoff=self.d_crit, box=box,
+                               return_distances=False)
+        if not self.exclusions is None:
+            pair = pair[~ _in2d(pair, self.exclusions)]


I'm surprised there was no suitable numpy operation for this :/

The below passes the analysis suite locally without using a custom Cython function. Maybe double check with Richard if his mixture of Cython/C++ is more efficient -- probably any advantage is related to the integer only nature of the data structure?

--- a/package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py +++ b/package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py @@ -210,6 +210,7 @@ from six import raise_from import numpy as np import scipy.optimize +from scipy.spatial.distance import cdist import warnings @@ -433,7 +434,7 @@ class HydrogenBondAutoCorrel(object): pair = capped_distance(self.h.positions, self.a.positions, max_cutoff=self.d_crit, box=box, return_distances=False) if not self.exclusions is None: - pair = pair[~ _in2d(pair, self.exclusions)] + pair = pair[~ (cdist(pair, self.exclusions)==0).any(axis=1)] hidx, aidx = np.transpose(pair)

You could use cdist but it's odd to use an O(n2) solution when you can just use a set

I don't have a strong view on it--if this is genuinely a performance-critical part of the code maybe the custom maintenance burden is worth it.

package/MDAnalysis/lib/_cutil.pyx

IAlibay · 2021-04-25T17:37:16Z

Forgot to mention, as far as I understand the old tests weren't picking up that this code path wasn't working. If that's the case, are we sure they are now working properly?

IAlibay

Thanks for the tests. For some reason codecov isn't recognising the ValueError test, just proposing a match to make sure it's actually picking up the right thing.

testsuite/MDAnalysisTests/lib/test_cutil.py

package/MDAnalysis/lib/_cutil.pyx

tylerjereddy · 2021-04-25T23:03:59Z

package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py

+        pair = capped_distance(self.h.positions, self.a.positions, max_cutoff=self.d_crit, box=box,
+                               return_distances=False)
+        if not self.exclusions is None:
+            pair = pair[~ _in2d(pair, self.exclusions)]


The below passes the analysis suite locally without using a custom Cython function. Maybe double check with Richard if his mixture of Cython/C++ is more efficient -- probably any advantage is related to the integer only nature of the data structure?

--- a/package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py +++ b/package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py @@ -210,6 +210,7 @@ from six import raise_from import numpy as np import scipy.optimize +from scipy.spatial.distance import cdist import warnings @@ -433,7 +434,7 @@ class HydrogenBondAutoCorrel(object): pair = capped_distance(self.h.positions, self.a.positions, max_cutoff=self.d_crit, box=box, return_distances=False) if not self.exclusions is None: - pair = pair[~ _in2d(pair, self.exclusions)] + pair = pair[~ (cdist(pair, self.exclusions)==0).any(axis=1)] hidx, aidx = np.transpose(pair)

tylerjereddy · 2021-04-25T23:13:25Z

package/MDAnalysis/lib/_cutil.pyx

+
+    for i in range(arr2.shape[0]):
+        p = pair[np.intp_t, np.intp_t](arr2[i, 0], arr2[i, 1])
+        hits.insert(p)


Not sure if we need a comment or two about the C++ magic happening here; I suppose we are slowly growing expertise with devs familiar with C++ (I'm not one of them).

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

richardjgowers · 2021-04-26T16:37:02Z

WRT codecov, I'm guessing Cython linenumbers and linetracing isn't a match made in heaven.

package/CHANGELOG

package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py

Co-authored-by: Oliver Beckstein <orbeckst@gmail.com>

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

IAlibay

lgtm!

richardjgowers added 4 commits April 25, 2021 12:17

added _in2d

be7b73a

removed deprecated numpy usage in hbond autocorrel

9d990f0

correct distances docs

5efe9a1

added C sources for updated _cutil

7f07893

richardjgowers requested a review from IAlibay April 25, 2021 16:34

IAlibay requested changes Apr 25, 2021

View reviewed changes

test for in2d

b46b200

IAlibay requested changes Apr 25, 2021

View reviewed changes

testsuite/MDAnalysisTests/lib/test_cutil.py Outdated Show resolved Hide resolved

package/MDAnalysis/lib/_cutil.pyx Show resolved Hide resolved

tylerjereddy reviewed Apr 25, 2021

View reviewed changes

richardjgowers and others added 3 commits April 26, 2021 15:45

Update testsuite/MDAnalysisTests/lib/test_cutil.py

42947d9

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

Update package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py

3004fa8

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

fixes from review

054117f

in2d tests match error message

898e151

IAlibay reviewed Apr 26, 2021

View reviewed changes

package/CHANGELOG Show resolved Hide resolved

IAlibay mentioned this pull request Apr 27, 2021

renamed 'pbc' kwarg to 'wrap' for Group methods #2857

Merged

4 tasks

orbeckst reviewed Apr 28, 2021

View reviewed changes

package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py Outdated Show resolved Hide resolved

richardjgowers and others added 3 commits April 28, 2021 08:19

changelog update

5cd83e1

Update package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py

fde050b

Co-authored-by: Oliver Beckstein <orbeckst@gmail.com>

Update package/MDAnalysis/analysis/hbonds/hbond_autocorrel.py

50531d9

Co-authored-by: Irfan Alibay <IAlibay@users.noreply.github.com>

IAlibay approved these changes Apr 28, 2021

View reviewed changes

IAlibay merged commit 76c9dc0 into master Apr 28, 2021

IAlibay mentioned this pull request May 2, 2021

HydrogenBondAutoCorrel uses deprecated features of numpy #2987

Closed

IAlibay deleted the hbond_excl_fix branch May 29, 2022 12:55

IAlibay added the defect label Sep 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hbond excl fix #3242

Hbond excl fix #3242

richardjgowers commented Apr 25, 2021

pep8speaks commented Apr 25, 2021 •

edited

codecov bot commented Apr 25, 2021 •

edited

IAlibay left a comment

IAlibay Apr 25, 2021

tylerjereddy Apr 25, 2021

richardjgowers Apr 26, 2021

tylerjereddy Apr 26, 2021 •

edited

IAlibay commented Apr 25, 2021

IAlibay left a comment

tylerjereddy Apr 25, 2021

tylerjereddy Apr 25, 2021

richardjgowers commented Apr 26, 2021

IAlibay left a comment

Hbond excl fix #3242

Hbond excl fix #3242

Conversation

richardjgowers commented Apr 25, 2021

Changes made in this Pull Request:

PR Checklist

pep8speaks commented Apr 25, 2021 • edited

Comment last updated at 2021-04-28 07:32:55 UTC

codecov bot commented Apr 25, 2021 • edited

Codecov Report

IAlibay left a comment

Choose a reason for hiding this comment

IAlibay Apr 25, 2021

Choose a reason for hiding this comment

tylerjereddy Apr 25, 2021

Choose a reason for hiding this comment

richardjgowers Apr 26, 2021

Choose a reason for hiding this comment

tylerjereddy Apr 26, 2021 • edited

Choose a reason for hiding this comment

IAlibay commented Apr 25, 2021

IAlibay left a comment

Choose a reason for hiding this comment

tylerjereddy Apr 25, 2021

Choose a reason for hiding this comment

tylerjereddy Apr 25, 2021

Choose a reason for hiding this comment

richardjgowers commented Apr 26, 2021

IAlibay left a comment

Choose a reason for hiding this comment

pep8speaks commented Apr 25, 2021 •

edited

codecov bot commented Apr 25, 2021 •

edited

tylerjereddy Apr 26, 2021 •

edited