Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement rank column in caprieval #213

Merged
merged 2 commits into from
Dec 21, 2021
Merged

Implement rank column in caprieval #213

merged 2 commits into from
Dec 21, 2021

Conversation

rvhonorato
Copy link
Member

This PR implements a new ranking column in caprieval and closes #212.

It adds two new parameters.

rankby = 'score'
rank_ascending = true

With the changes on this PR its possible to use one column to do the ranking and another for the sorting, let me know if this is not the desired behavior of the feature.

@rvhonorato
Copy link
Member Author

For example:

sortby = 'irmsd'
rankby = 'score'
             model                   rank     score     irmsd      fnat     lrmsd    ilrmsd
01_rigidbody/rigidbody_15.pdb           7   -22.486     1.367     0.500     3.158     2.285
01_rigidbody/rigidbody_4.pdb            1   -33.049     1.621     0.389     3.589     2.908
01_rigidbody/rigidbody_9.pdb           17   -13.026     2.549     0.417     7.081     4.720
01_rigidbody/rigidbody_16.pdb           3   -24.729     2.950     0.361     7.472     4.502
01_rigidbody/rigidbody_20.pdb          10   -20.785     3.289     0.222     6.922     4.261
01_rigidbody/rigidbody_6.pdb           20    -5.208     3.542     0.250    13.377     6.768
01_rigidbody/rigidbody_18.pdb          18   -12.457     3.994     0.194    10.660     6.087
01_rigidbody/rigidbody_2.pdb           16   -14.105     4.628     0.111     9.452     6.978
01_rigidbody/rigidbody_12.pdb           9   -21.579     5.421     0.111    11.473     8.563
01_rigidbody/rigidbody_8.pdb            6   -22.546     5.992     0.083    11.293     9.180
01_rigidbody/rigidbody_13.pdb          11   -17.550     7.714     0.056    14.697    12.237
01_rigidbody/rigidbody_10.pdb           4   -24.432     8.241     0.111    16.298    15.680
01_rigidbody/rigidbody_14.pdb          12   -15.512     8.381     0.083    15.870    13.580
01_rigidbody/rigidbody_19.pdb           8   -21.948     8.630     0.167    19.544    16.463
01_rigidbody/rigidbody_3.pdb            2   -25.237     8.723     0.056    18.450    14.583
01_rigidbody/rigidbody_7.pdb           14   -14.964     9.421     0.028    19.315    14.640
01_rigidbody/rigidbody_5.pdb           13   -15.457    10.007     0.111    20.587    17.136
01_rigidbody/rigidbody_11.pdb           5   -22.866    10.012     0.111    18.592    18.318
01_rigidbody/rigidbody_17.pdb          15   -14.898    10.089     0.111    17.690    17.885
01_rigidbody/rigidbody_1.pdb           19    -9.785    10.827     0.028    26.500    18.997

sort_ascending = true
# which field should be used to create the ranking
rankby = 'score'
rank_ascending = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen if both sort_ascending and rank_ascending are true and sortby is set to e.g. i-rmsd while rankby is set by score?

Shouldn't we have only sort_ascending?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen if both sort_ascending and rank_ascending are true and sortby is set to e.g. i-rmsd while rankby is set by score?

This scenario is the example I posted above

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But aren't sort_ascending and rank_ascending redundant??? I don't see the need for the rank_ascending parameter

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one defined the direction in which the table will be sorted, the other the direction of the value that we will rank the models;
for sorting: irmsd we'd sort asceding (the lower the best), fnat descending (the higher the best)
for ranking: haddock-score ascending (the lower the best), some other module in which the higher the best = descending

But up to you, I can remove it as well. Rule # 1 !

@amjjbonvin
Copy link
Member

amjjbonvin commented Dec 21, 2021 via email

@rvhonorato
Copy link
Member Author

rvhonorato commented Dec 21, 2021

Currently the ranking is independent from the sorting, it can be ranked on one parameter and sorted on another, and in different directions.

Is it better to make it so that the table will be ranked and sorted to only one parameter?

@amjjbonvin
Copy link
Member

amjjbonvin commented Dec 21, 2021 via email

@rvhonorato rvhonorato merged commit 5aff5f5 into main Dec 21, 2021
@rvhonorato rvhonorato deleted the 212 branch December 21, 2021 17:15
@joaomcteixeira joaomcteixeira added the feature New feature request label Dec 22, 2021
@joaomcteixeira joaomcteixeira added this to In Progress in Features via automation Dec 22, 2021
@joaomcteixeira joaomcteixeira moved this from In Progress to Done in Features Dec 22, 2021
@joaomcteixeira joaomcteixeira added the m|caprieval Improvements in caprieval module label Jan 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature request m|caprieval Improvements in caprieval module
Projects
Features
  
Done
Development

Successfully merging this pull request may close these issues.

Add rank to the caprieval module output
3 participants