Integration tests #281

pckroon · 2020-06-26T15:04:20Z

Sets up integration tests based on files provided by @paulocts

Fixes #183

Will need cleanup and rebasing before it's ready for review.

fgrunewald · 2020-07-21T12:38:23Z

two small comments I noticed when doing the EN, which might be relevant here as well:

direct comparison of interactions won't work because martinize has these ifdef statements (i.e. meta is populated differently) and also the force-constant is set as variable
mapping of martinize1 and martinize2 is different so EN will not be the same. Generally for the martini2 stuff martinize1 is taken as reference as far as I understand

Update integration test reference files

With the same py interpreter used to run the tests, and find Martinize2 in PATH, to make sure it's installed.

…eters

Update tier-1 tests Update tier-1/bpti files

pckroon · 2020-08-10T14:07:29Z

I rebased it to clean up the history a bit. It still requires 1 file (tier-1/bpti/martinize2/molecule_0.itp) to be checked by paulo

jbarnoud

Here is a first round with the datafiles. I do not want to take the risk of loosing these comments is github fucks up.

vermouth/tests/data/integration_tests/tier-0/mini-protein1_betasheet/martinize1/command

vermouth/tests/data/integration_tests/tier-0/mini-protein2_helix/README

vermouth/tests/data/integration_tests/tier-0/mini-protein1_betasheet/README

vermouth/tests/data/integration_tests/tier-0/mini-protein2_helix/README

vermouth/tests/data/integration_tests/tier-0/mini-protein2_helix/martinize2/command

vermouth/tests/data/integration_tests/tier-1/bpti/martinize2/command

vermouth/tests/data/integration_tests/tier-1/lysozyme/README

vermouth/tests/data/integration_tests/tier-1/lysozyme/martinize2/command

vermouth/tests/data/integration_tests/tier-1/lysozyme/test/CG.top

vermouth/tests/data/integration_tests/tier-1/villin/README

pckroon · 2020-08-21T14:41:02Z

Turns out the cg.pdb does not get written to the current working dir, but somewhere else. Investigating...

vermouth/tests/integration_tests/test_integration.py

jbarnoud · 2020-08-21T17:13:15Z

vermouth/tests/integration_tests/test_integration.py

+MARTINIZE2 = find_in_path()
+
+
+def assert_equal_blocks(block1, block2):


Wha about https://docs.pytest.org/en/stable/assert.html#defining-your-own-explanation-for-failed-assertions ?

I just tried, and it became a mess: (pytest_assertion plugin: representation of details failed: .../lib/python3.6/site-packages/_pytest/assertion/util.py:377: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all().. The only way to make it work would be to basically reimplement the pytest reporting machinery, which seems rather out-of-scope.
Similarly for the 'distance' attribute on edges, and interactions, these seem plagued by rounding issues, and I can't figure out a way to make the output remotely useful.

Oh, adding a custom reporter for np.ndarray does not work. You'd need to get into the assertion rewriting, which is not possible pytest-dev/pytest#5347

vermouth/tests/integration_tests/test_integration.py

jbarnoud · 2020-08-22T08:08:40Z

I do not find what you do with the files in the "martinize1" directories. Are you using them or did I miss something? Also, I think it would be good to separate what is expected to be the same between martinize 1 and 2 , and what is expected to be different. This way, you can make sure martinize 2 produces the same output as martinize 1 when it should.

This way the output pdb file ends up at the correct place: the temporary directory. To enable rewriting to the data file path (e.g. for reading), prepend a '.'

pckroon · 2020-08-24T13:01:48Z

I removed the M1 directories. They were not used in the tests, and only in the hand crafting of the files in the M2 directories. Those files have been carefully vetted.
I don't see a feasible way of creating a custom pytest reporter for Molecule that gives me remotely comparable output to what I have now.

- the first colinear dummy was always placed on the anchor - Fix approx testing of coordinates in integration tests - Fix approx testing of distances of edges in integration tests - Remove positions from charge dummy nodes in integration tests, since those are (partially) random. The relative positions are tests by edge distances.

The removed rubber band got a distance of 0.92, which is more than the maximum (0.9)

1e-3 is the precision of PDB files

pckroon · 2020-08-25T10:05:33Z

The low patch coverage is kind of expected, since the integration tests run in a separate interpreter, and the coverage they generate is not recorded.

fgrunewald

Overall looks good to me. Few small comments double checking some things. Also before minting the new release we should include all interactions in the ff-parser and itp parser.

bin/martinize2

vermouth/data/force_fields/elnedyn22/modifications.ff

vermouth/data/mappings/elnedyn/elnedyn22/modifications.mapping

vermouth/data/mappings/martini30/modifications.mapping

vermouth/processors/locate_charge_dummies.py

pckroon · 2020-09-03T09:07:01Z

Also before minting the new release we should include all interactions in the ff-parser and itp parser.

Disagree. A release is long overdue, and using not-yet-implemented interactions causes an understandable error message.

fgrunewald

Breaking out comments to issues this is good to go.

pckroon · 2020-09-03T09:46:40Z

Created #299 #300 #301

pckroon closed this Jun 26, 2020

pckroon reopened this Jun 26, 2020

pckroon marked this pull request as draft June 26, 2020 15:05

pckroon and others added 11 commits August 10, 2020 15:50

Sort atoms in writing conect records

cae2921

Correct typo in datafiles license

6454195

Initial tier0 tests

ac21a66

Update integration test reference files

Implement pdb and itp comparison by parsing

8279952

Make integration tests run more reliably

6ee687c

With the same py interpreter used to run the tests, and find Martinize2 in PATH, to make sure it's installed.

Smarter handling of command, fuzzy comparison for interaction param…

d96c18b

…eters

Sort interactions before comparing in integration tests

c664cf6

Change CLI defaults for elastic network to be the same as martinize1

f39d469

Add tier1 tests

f1cf328

Update tier-1 tests Update tier-1/bpti files

Add ASP0 modification + mapping to elnedyn22

223a6cb

Make vermouth.gmx expose read_itp

0775af6

pckroon force-pushed the integration_tests branch from 3582f25 to 0775af6 Compare August 10, 2020 14:04

Py35 compatability

8fa84c4

pckroon marked this pull request as ready for review August 18, 2020 10:58

pckroon requested a review from jbarnoud August 18, 2020 10:59

jbarnoud reviewed Aug 21, 2020

View reviewed changes

Peter Kroon added 5 commits August 21, 2020 15:42

Merge remote-tracking branch 'origin/master' into integration_tests

f021458

Fix warning counter

df3cc22

Address most data comments

b6dbee3

Tighten tests --- ensure files exist

4d2119b

Change 'step' to more correct 'model'

2d9523f

jbarnoud reviewed Aug 21, 2020

View reviewed changes

Don't rewrite cg.pdb to data files

c350a3a

This way the output pdb file ends up at the correct place: the temporary directory. To enable rewriting to the data file path (e.g. for reading), prepend a '.'

Peter Kroon added 6 commits August 24, 2020 15:39

Put back vetted cg.pdb files that got overwritten by accident

8ee7eff

Protein termini for martini30 FFs

ed9621a

Change rubber bands due to coordinate change from termini mapping

87262d8

The removed rubber band got a distance of 0.92, which is more than the maximum (0.9)

Change scfix dihedral phase due to mapping change of termini

ca783d5

Change position of terminus due to mapping

1fbc1c1

pckroon requested a review from jbarnoud August 24, 2020 14:51

Peter Kroon and others added 2 commits August 25, 2020 11:52

Approx abs tol to 1e-3 for positions in integration tests

b0ba242

1e-3 is the precision of PDB files

Merge branch 'master' into integration_tests

c2a1d5d

pckroon requested a review from fgrunewald August 26, 2020 09:52

fgrunewald requested changes Sep 2, 2020

View reviewed changes

fgrunewald approved these changes Sep 3, 2020

View reviewed changes

pckroon merged commit c4471b4 into master Sep 3, 2020

pckroon deleted the integration_tests branch September 3, 2020 09:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration tests #281

Integration tests #281

pckroon commented Jun 26, 2020 •

edited

fgrunewald commented Jul 21, 2020

pckroon commented Aug 10, 2020

jbarnoud left a comment

pckroon commented Aug 21, 2020

jbarnoud Aug 21, 2020

pckroon Aug 24, 2020

pckroon Aug 24, 2020

jbarnoud commented Aug 22, 2020

pckroon commented Aug 24, 2020

pckroon commented Aug 25, 2020

fgrunewald left a comment

pckroon commented Sep 3, 2020

fgrunewald left a comment

pckroon commented Sep 3, 2020

		MARTINIZE2 = find_in_path()


		def assert_equal_blocks(block1, block2):

Integration tests #281

Integration tests #281

Conversation

pckroon commented Jun 26, 2020 • edited

fgrunewald commented Jul 21, 2020

pckroon commented Aug 10, 2020

jbarnoud left a comment

Choose a reason for hiding this comment

pckroon commented Aug 21, 2020

jbarnoud Aug 21, 2020

Choose a reason for hiding this comment

pckroon Aug 24, 2020

Choose a reason for hiding this comment

pckroon Aug 24, 2020

Choose a reason for hiding this comment

jbarnoud commented Aug 22, 2020

pckroon commented Aug 24, 2020

pckroon commented Aug 25, 2020

fgrunewald left a comment

Choose a reason for hiding this comment

pckroon commented Sep 3, 2020

fgrunewald left a comment

Choose a reason for hiding this comment

pckroon commented Sep 3, 2020

pckroon commented Jun 26, 2020 •

edited