Issue 68 edesolv

[DarioMarzella]

Energy of desolvation currently working in this branch (complete issue: #issue68).
The organization of deeprank/features/Edesolv.py though might not be great, so I would appreciate comments on that (if needed).

[coveralls]

Pull Request Test Coverage Report for Build 1127779947

• 133 of 166 (80.12%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.2%) to 77.329%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
deeprank/features/Edesolv.py	131	164	79.88%

Totals
Change from base Build 741819231:	0.2%
Covered Lines:	1760
Relevant Lines:	2276

---

💛 - Coveralls

[CunliangGeng]

Please leave a comment about the equations and references used for calculating Edesolv.

[DarioMarzella]

Do you mean in the class docstring?

[CunliangGeng]

Yep

[DarioMarzella]

Improved in 6fed0fc . Please let me know if it's not satisfactory yet.

[CunliangGeng]

Split the chains to independent pdbs and use those for SA calculation of free chains

[DarioMarzella]

Wouldn't that just cause extra unnecessary I/O (or memory) usage? Can I ask you why this is not fine?

[CunliangGeng]

The extra memory usage is quite cheap.
The main issue here is your method cannot make sure the chains are really away from each other, and also it will cause troubles when applied to multiple-chain complexes.

[DarioMarzella]

Makes sense. Will change it.

[CunliangGeng]

The loop here is using the contact atoms to calculate Edsolv. The contact atoms are dependent on distance cutoff. So the Edsolv will also be dependent on distance cutoff. It is not a great idea to calculate Edsolv in this way.
You could keep these code and rewrite one as the way used in Haddock, then compare the results to check the difference.

[DarioMarzella]

So would you want me to calculate the Edesolv for each atom in the proteins (as HADDOCK does), although in the end we only use the ones in the grid? Do you want it to be done constitutively or just for the comparison with HADDOCK?

[CunliangGeng]

Aha, I forgot we need only the atoms in the grid. It's fine then to calculate the Edesolv for only contact atoms with one place to improve:
Make sure the distance_cutoff for defining contact is same as the one used to get grid atoms. You should transfer the value in DataGenerator.create_database(contact_distance=8.5) to the "# get the contact atoms" step in this code.

To make sure the results are correct, you also need to compare the results with Haddock by giving a big enough distance cutoff to get all atoms.

[DarioMarzella]

Ok.
Thank you very much for the tips, will do.

[CunliangGeng]

Nice work, thanks.
I summarise the major comments below:

1. Try to calculate the Edsolv as how Haddock does and get rid of the dependence on contact atoms
2. Try to use absolute difference rather than pearson correlation coefficient to compare the results between the code and Haddock
3. Write an unit test to test the code

[DarioMarzella]

3. Write an unit test to test the code

For point 3, isn't it better if I simply add Edesolv as a feature in test_generate.py? In this way, it will be tested along with the other features and there will be no need to change its specific unit test.

[CunliangGeng]

1. Write an unit test to test the code

For point 3, isn't it better if I simply add Edesolv as a feature in test_generate.py? In this way, it will be tested along with the other features and there will be no need to change its specific unit test.

Yep, makes sense :-)

[CunliangGeng]

@DarioMarzella Please leave me a message when all requested changes have been handled :-)
The tests failed and please look into it.

[DarioMarzella]

@DarioMarzella Please leave me a message when all requested changes have been handled :-)
The tests failed and please look into it.

Yes, I am waiting for the comparison with HADDOCK to update you :)

[DarioMarzella]

@DarioMarzella Please leave me a message when all requested changes have been handled :-)
The tests failed and please look into it.

I just got the result, and seems like the Edesolv I get from DeepRank by setting a grid big enough to take the whole protein correlates a bit worse with haddock than the one I previously got by taking into account the interface only. This makes me wonder if HADDOCK is using the whole proteins or the surface atoms only for the Edesolv.
(Please note I had to use a different set of structures previously used for comparison. I previously used all the CAPRI set structures, now I had to use only some BM5 structures)

Last result (using all-atom Edesolv):
Pearson correlation: 0.92
Mean absolute difference: 4.13
Plot:
haddock_vs_deeprank.pdf

Previous results (using interface-only Edesolv):
Pearson correlation: 0.98
Mean absolute difference: -2.93
Plot:
haddock_vs_deeprank_capri.pdf

[DarioMarzella]

Update: it might be that HADDOCK uses different thresholds to calculate the Edesolv.
My values correlate better with HADDOCK itw and worse with HADDOCK it1 and it0.

HADDOCK it0 models:
Pearson corr: 0.89
Mean Absolute difference: 5.05
Plot:

HADDOCK it1 models:
Pearson: 0.95
Mean Absolute Difference: 2.98
Plot:

HADDOCK itw models:
Pearson: 0.97
Mean Absolute value: 3.13
Plot:

[CunliangGeng]

@DarioMarzella Thanks for the results! If we're using the same equations and parameters as Haddock, the results should be same or the difference should be at the level of precision, e.g. 0.01(?). Did you check the Haddock code? If not, you'd better to have a look and also get clear with your wonder "This makes me wonder if HADDOCK is using the whole proteins or the surface atoms only for the Edesolv." (I don't remember and cannot provide a direct answer here :-(

And note the difference might be also from the calculation of surface area, Haddock does not use biopython for that. If the major part of difference is from area calculation, it's fine then and we should leave a note in the documentation.

[CunliangGeng]

This repo is going to be archived #260. So closing this PR now.