Create `haddock3-score` CLI #510

rvhonorato · 2022-07-25T12:35:04Z

~~I deleted the PR template since this is an extra tool and not a module/gear/libs/core/clis/cns~~

~~Adds a tool to calculate the haddock-score of a complex.~~

~~I see quite some more things that can be implemented such as support for pdb.list, custom-weights, use restraints, etc - we can open additional issues for that.~~

~~(venv) repos/haddock3 » pwd~~
~~/Users/rodrigo/repos/haddock3~~
~~(venv) repos/haddock3 » export PYTHONPATH=pwd/src~~
(venv) repos/haddock3 » time python tools/haddock-score.py examples/data/2oob.pdb
HADDOCK-score (emscoring): -53.8110
~~python tools/haddock-score.py examples/data/2oob.pdb 2.32s user 0.08s system 97% cpu 2.477 total~~

@joaomcteixeira:

Add a new command-line using the libworkflow machinery to run emscoring directly on a complex (PDB file) using the command line.
Idea was develop by @rvhonorato and @amjjbonvin
@joaomcteixeira worked out the integration in haddock. See past commits for details.~

Cheers,

amjjbonvin

Would it be an idea to add a few options to:

Possibly printout also the components of the score
Save the generated PDB file (now only kept into memory

This command should in principle return a score rather similar to the EM scoring example, right?

mgiulini

User experience is great! but I agree that having the different components would be useful.

I calculated the haddock-score for the first 20 pdbs of 1_emscoring for capri Target218 (check out on tintin if you like). It is highly correlated to the true haddock score, but the pearson coeff. is not super (r~0.88). Should we add more steps to this? Or is it ok as it is?

amjjbonvin · 2022-07-25T15:20:18Z

Where does the difference come from… that’s the question. The number of EM steps in emscoring in only 50 while it is 200 in emref And I assume autohis is true for both?

rvhonorato · 2022-07-25T16:29:59Z

The number of EM steps in emscoring in only 50 while it is 200 in emref And I assume autohis is true for both?

Yep, this tool runs emscoring under the hood

And I assume autohis is true for both?

This tool uses the default parameters

rvhonorato · 2022-07-25T16:35:10Z

Possibly printout also the components of the score

Done

Save the generated PDB file (now only kept into memory

It is saved as calc-hs_1.pdb in the directory that you ran the command

amjjbonvin · 2022-07-25T18:39:43Z

Possibly printout also the components of the score Done

As an option?

Save the generated PDB file (now only kept into memoryI

I would rather save it as <original-pdb-filename)_hs.pdb to avoid overwriting it if running on multiple PDBs in the same dir.

amjjbonvin · 2022-07-25T19:07:32Z

tools/haddock-score.py

+    print("-----")
+    print(f"vdw\t{vdw:.4f}")
+    print(f"elec't{elec:.4f}")
+    print(f"desolv\t{desolv:.4f}")
+    print(f"air\t{air:.4f}")
+    print("HADDOCK-score = (1.0 * vdw) + (0.2 * elec) + (1.0 * desolv) + (0.1 * air)")
+    print("-----")
+    print(f"HADDOCK-score (emscoring) = {haddock_score_itw:.4f}")


air will be in principle 0
may-be more interesting to output the BSA.
And I would not output everything by default, but only as an option (e.g. if the -full argument is given).
Simpler for incorporating the score in other scripts.

And same thing for outputting the pdb and psf files, only if an option is given, e.q. -outpdb -outpsf

air will be in principle 0
may-be more interesting to output the BSA.
And I would not output everything by default, but only as an option (e.g. if the -full argument is given).
Simpler for incorporating the score in other scripts.

Added --full option

And same thing for outputting the pdb and psf files, only if an option is given, e.q. -outpdb -outpsf

Added --outputpdb and --outputpsf options

joaomcteixeira · 2022-07-25T23:12:45Z

Hi,
Quick question before a more in-depth review. Why can't this be a CLI?
cheers

joaomcteixeira · 2022-07-25T23:14:01Z

.github/workflows/tests.yml

@@ -48,5 +48,5 @@ jobs:
      uses: codecov/codecov-action@v2
      with:
        files: ./coverage.xml
-        fail_ci_if_error: true
+        fail_ci_if_error: false


Why change this to false?

codecov-commenter · 2022-07-26T07:28:14Z

Codecov Report

Merging #510 (5c15203) into main (de997e4) will decrease coverage by 1.44%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #510      +/-   ##
==========================================
- Coverage   74.84%   73.40%   -1.45%     
==========================================
  Files         105      103       -2     
  Lines        6953     6621     -332     
==========================================
- Hits         5204     4860     -344     
- Misses       1749     1761      +12

Impacted Files	Coverage Δ
src/haddock/modules/topology/topoaa/__init__.py	`44.44% <0.00%> (+0.37%)`	⬆️
src/haddock/libs/libfunc.py	`80.95% <0.00%> (-9.53%)`	⬇️
src/haddock/modules/__init__.py	`73.37% <0.00%> (-3.36%)`	⬇️
tests/test_modules_general.py	`97.08% <0.00%> (-0.56%)`	⬇️
examples/compare_runs.py	`23.77% <0.00%> (-0.51%)`	⬇️
src/haddock/libs/libworkflow.py	`30.13% <0.00%> (-0.42%)`	⬇️
src/haddock/gear/prepare_run.py	`50.77% <0.00%> (-0.01%)`	⬇️
tests/test_libutil.py	`100.00% <0.00%> (ø)`
tests/test_gear_config_writer.py
src/haddock/gear/config_reader.py
... and 7 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

rvhonorato · 2022-07-26T07:28:37Z

I would rather save it as <original-pdb-filename)_hs.pdb to avoid overwriting it if running on multiple PDBs in the same dir.

Done

rvhonorato · 2022-07-26T07:31:25Z

I calculated the haddock-score for the first 20 pdbs of 1_emscoring for capri Target218 (check out on tintin if you like). It is highly correlated to the true haddock score, but the pearson coeff. is not super (r~0.88). Should we add more steps to this? Or is it ok as it is?

We figured out that the source of the difference is that @mgiulini was calculating the haddock-scores in the already-minimized models instead of the input ones.

joaomcteixeira · 2022-07-26T07:37:02Z

MANIFEST.in

@@ -7,6 +7,8 @@ include LICENSE
 include requirements.txt
 include requirements.yml

+recursive-include tools *.py


I disagree here. I strongly suggest making haddock-score a CLI.

joaomcteixeira · 2022-07-26T08:02:38Z

tools/haddock-score.py

+            for line in tidy_pdbfile(inp_fh):
+                out_fh.write(line)


Single call to save to disk

Suggested change

for line in tidy_pdbfile(inp_fh):

out_fh.write(line)

lines = list(tidy_pdbfile(inp_fh))

out_fh.write(os.linesep.join(lines))

joaomcteixeira · 2022-07-26T08:02:52Z

tools/haddock-score.py

+"""
+
+import argparse
+import subprocess


Suggested change

import subprocess

import os

import subprocess

joaomcteixeira · 2022-07-26T08:03:34Z

tools/haddock-score.py

+    main_topoaa_cns_script_as_string = Path(
+        topoaa_module_folder, "cns/generate-topology.cns"
+        ).read_text()


Suggested change

main_topoaa_cns_script_as_string = Path(

topoaa_module_folder, "cns/generate-topology.cns"

).read_text()

main_topoaa_cns_script_as_string = Path(

topoaa_module_folder,

"cns",

"generate-topology.cns"

).read_text()

joaomcteixeira · 2022-07-26T08:04:27Z

tools/haddock-score.py

+    main_emscoring_cns_script_as_string = Path(
+        emscoring_module_folder, "cns/emscoring.cns"
+        ).read_text()


Suggested change

main_emscoring_cns_script_as_string = Path(

emscoring_module_folder, "cns/emscoring.cns"

).read_text()

main_emscoring_cns_script_as_string = Path(

emscoring_module_folder,

"cns",

"emscoring.cns",

).read_text()

joaomcteixeira

I just made small code suggestions. I think the topo and emscoring wrappers are a great opportunity to try to use the module objects directly, or at least to study and learn why we can't yet (if that's the case) use them here already. Anyway, that can be for another PR if there's rush merging this one.

Finally, I really think we should have haddock-score as a client as all that is necessary is already placed.

Cheers,

joaomcteixeira · 2022-08-10T11:05:30Z

Notethis is not ready to merge. I made an edit in the topoaa I need to confirm it does not affect the workflows. Maybe it still needs some additional work internally.

mgiulini · 2022-08-10T11:07:21Z

Notethis is not ready to merge. I made an edit in the topoaa I need to confirm it does not affect the workflows. Maybe it still needs some additional work internally.

sure! I was testing it and noticed a couple of improvements :)

…docking/haddock3 into 365-implement-haddock3-score

joaomcteixeira · 2022-08-10T12:52:50Z

@mgiulini can you give a look now, also to the new parameter -p? Let me know if the program outputs the values you are expecting. Cheers

joaomcteixeira · 2022-08-16T10:18:01Z

All workflows perform correctly despite the change in topoaa. Everything looks good. 👍

rvhonorato · 2022-08-21T05:36:14Z

I was testing it and found a mini-bug. I wonder if it makes sense to give the user the possibility to choose the number of energy minimization steps (nemsteps) directly from the cli..In my case study it would be useful

Don't think this makes sense, the idea is to calculate the haddock score according to the defined weights, not use the calculation routine to do minimisation steps.

I know it's doable but in my opinion it falls out of scope, sounds like this is the use case for the default workflow pipeline.

amjjbonvin · 2022-08-21T07:56:26Z

There is however a minimization step and I think it is nice to be able to control the number of steps. Even set it to 0 if you really want the score of the “raw” model

rvhonorato · 2022-08-21T15:57:19Z

But then we need to make this very clear, imagine someone is trying to recalculate the haddock score of a given complex and the numbers are different because the minimisation steps were not the default.

Being able to customize the steps is nice indeed, I'm just thinking about reproducibility.

Maybe print it out in big letter that's the steps were altered (and also the weights if that's the case) and that this information needs to accompany the publication.

(Or leave this option out and let the user use the default workflow running scheme instead)

amjjbonvin · 2022-08-21T18:03:43Z

My guess is most users will simply use the default settings

mgiulini · 2022-08-22T08:24:46Z

Hi there! I agree with @rvhonorato that the use of non-default parameters is dangerous and that this should be made very clear during the execution. I created a couple of warnings for that. If you agree l will give the possibility to modify the weights.

joaomcteixeira · 2022-08-22T10:58:28Z

To help solving the discussion, the haddock3-score CLI will use the default values unless otherwise specified. Currently, users can modify any parameter of the emscore using the -p option, for example:

haddock3-score complex.pdb -p nemsteps 50 w_air 1 electflag True

See more with haddock3-score -h.

For reporting purposes, users can always save the command line use.

I am okay with the warning messages proposed by @mgiulini .

For me, it is okay to be merged. 👍 Good work everyone 🚀

mgiulini · 2022-08-30T08:52:09Z

hello, I changed a bit the code to consider (possibly) new weights in the calculation of the score.

joaomcteixeira · 2022-08-30T09:03:18Z

I cant test or approve now because i am on mobile. Looks great and the change was necessary because previously if users injected the weights they were not being used and now they are. Great work! El mar., 30 ago. 2022 10:52, Marco Giulini ***@***.***> escribió:

…

@mgiulini <https://github.com/mgiulini> requested your review on: #510 <#510> Create haddock3-score CLI. — Reply to this email directly, view it on GitHub <#510 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWEOMX3NU6QXOKM64B546DV3XDUXANCNFSM54SEAVRQ> . You are receiving this because your review was requested.Message ID: ***@***.***>

joaomcteixeira

Sounds good @mgiulini 👍

src/haddock/clis/cli_score.py

Create haddock-score.py

508302d

rvhonorato requested review from amjjbonvin and mgiulini July 25, 2022 12:35

rvhonorato self-assigned this Jul 25, 2022

rvhonorato linked an issue Jul 25, 2022 that may be closed by this pull request

Implement haddock3-score #365

Closed

rvhonorato added 2 commits July 25, 2022 14:44

Update MANIFEST.in

c21c355

Update tests.yml

c5da365

amjjbonvin reviewed Jul 25, 2022

View reviewed changes

mgiulini reviewed Jul 25, 2022

View reviewed changes

Update haddock-score.py

90ee034

rvhonorato requested review from amjjbonvin and mgiulini July 25, 2022 16:36

amjjbonvin reviewed Jul 25, 2022

View reviewed changes

joaomcteixeira reviewed Jul 25, 2022

View reviewed changes

Update haddock-score.py

10f9cd0

rvhonorato requested a review from amjjbonvin July 26, 2022 07:28

joaomcteixeira reviewed Jul 26, 2022

View reviewed changes

joaomcteixeira added 6 commits August 10, 2022 14:01

Merge branch 'main' into 365-implement-haddock3-score

2754485

protoclean topoaa

286faab

Merge branch '365-implement-haddock3-score' of https://github.com/had…

80370ff

…docking/haddock3 into 365-implement-haddock3-score

corrections and addresses comments

0959b73

update docs

f228f52

update docs

dc205c4

joaomcteixeira added 2 commits August 16, 2022 11:19

completes the client and solves some issues

1634e5e

add CLI doc page

7fbf015

joaomcteixeira requested a review from mgiulini August 16, 2022 10:18

added warning messages

ba4a1da

joaomcteixeira approved these changes Aug 22, 2022

View reviewed changes

added suppot of different weights

5c15203

mgiulini requested a review from joaomcteixeira August 30, 2022 08:52

joaomcteixeira approved these changes Aug 30, 2022

View reviewed changes

src/haddock/clis/cli_score.py Outdated Show resolved Hide resolved

added ATTENTION messages

3549dc4

mgiulini merged commit d19bbf6 into main Aug 31, 2022

mgiulini deleted the 365-implement-haddock3-score branch August 31, 2022 08:13

mgiulini mentioned this pull request Nov 25, 2022

check values in cfg files against .yaml files #487

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create `haddock3-score` CLI #510

Create `haddock3-score` CLI #510

rvhonorato commented Jul 25, 2022 •

edited

Loading

amjjbonvin left a comment

mgiulini left a comment

amjjbonvin commented Jul 25, 2022 via email

rvhonorato commented Jul 25, 2022

rvhonorato commented Jul 25, 2022

amjjbonvin commented Jul 25, 2022 via email

amjjbonvin Jul 25, 2022

amjjbonvin Jul 25, 2022

rvhonorato Jul 26, 2022

joaomcteixeira commented Jul 25, 2022

joaomcteixeira Jul 25, 2022

codecov-commenter commented Jul 26, 2022 •

edited

Loading

rvhonorato commented Jul 26, 2022

rvhonorato commented Jul 26, 2022

joaomcteixeira Jul 26, 2022

joaomcteixeira Jul 26, 2022

joaomcteixeira Jul 26, 2022

joaomcteixeira Jul 26, 2022

joaomcteixeira Jul 26, 2022

joaomcteixeira left a comment

joaomcteixeira commented Aug 10, 2022

mgiulini commented Aug 10, 2022

joaomcteixeira commented Aug 10, 2022

joaomcteixeira commented Aug 16, 2022

rvhonorato commented Aug 21, 2022

amjjbonvin commented Aug 21, 2022 via email

rvhonorato commented Aug 21, 2022 •

edited

Loading

amjjbonvin commented Aug 21, 2022 via email

mgiulini commented Aug 22, 2022

joaomcteixeira commented Aug 22, 2022

mgiulini commented Aug 30, 2022

joaomcteixeira commented Aug 30, 2022 via email

joaomcteixeira left a comment

Create haddock3-score CLI #510

Create haddock3-score CLI #510

Conversation

rvhonorato commented Jul 25, 2022 • edited Loading

amjjbonvin left a comment

Choose a reason for hiding this comment

mgiulini left a comment

Choose a reason for hiding this comment

amjjbonvin commented Jul 25, 2022 via email

rvhonorato commented Jul 25, 2022

rvhonorato commented Jul 25, 2022

amjjbonvin commented Jul 25, 2022 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joaomcteixeira commented Jul 25, 2022

Choose a reason for hiding this comment

codecov-commenter commented Jul 26, 2022 • edited Loading

Codecov Report

rvhonorato commented Jul 26, 2022

rvhonorato commented Jul 26, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joaomcteixeira left a comment

Choose a reason for hiding this comment

joaomcteixeira commented Aug 10, 2022

mgiulini commented Aug 10, 2022

joaomcteixeira commented Aug 10, 2022

joaomcteixeira commented Aug 16, 2022

rvhonorato commented Aug 21, 2022

amjjbonvin commented Aug 21, 2022 via email

rvhonorato commented Aug 21, 2022 • edited Loading

amjjbonvin commented Aug 21, 2022 via email

mgiulini commented Aug 22, 2022

joaomcteixeira commented Aug 22, 2022

mgiulini commented Aug 30, 2022

joaomcteixeira commented Aug 30, 2022 via email

joaomcteixeira left a comment

Choose a reason for hiding this comment

Create `haddock3-score` CLI #510

Create `haddock3-score` CLI #510

rvhonorato commented Jul 25, 2022 •

edited

Loading

codecov-commenter commented Jul 26, 2022 •

edited

Loading

rvhonorato commented Aug 21, 2022 •

edited

Loading