Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mhcnuggets scores on TESLA samples #23

Open
vladimirkovacevic opened this issue Dec 3, 2020 · 4 comments
Open

mhcnuggets scores on TESLA samples #23

vladimirkovacevic opened this issue Dec 3, 2020 · 4 comments

Comments

@vladimirkovacevic
Copy link

vladimirkovacevic commented Dec 3, 2020 •
edited
Loading

I was very curious to test the performance of mhcnuggets 2.3 on the latest dataset with TESLA validated neoantigen candidates published in Cell (Key Parameters of Tumor Epitope Immunogenicity
Revealed Through a Consortium Approach Improve
Neoantigen Prediction, Table S4 and S7). In TESLA they used flow cytometry and microscopy to confirm (VALIDATE) neoantigen candidates. More info is available in the paper.
Here are ic50 scores obtained by mhcnuggets.
mhcnuggets_validated
The distributions of confirmed and not confirmed candidates are in a very similar range:
Comfirmed candidates(041): Median=5598.78, Mean=7441.88(+-9090.58)
Not confirmed candidates(871): Median=5605.33, Mean=9153.18(+-10031.29)
mhcnuggets_boxplot
When mhcnuggets scores are normalized (with added minus sign since smaller ic50 is better) and compared against VALIDATED AUC score is 0.513, precision is 0.0596, here is the output comparison figure:
mhcnuggets_plot

Here is the table with obtained scores as a reference and command used for one of the mhcnuggets runs:
python /mhcnuggets/mhcnuggets/src/predict.py -c I --allele HLA-C*05:01 --peptides merged.fasta -o mhcnuggets_HLA-C*05:01.peps

Are these results expected? Could it be that this use case is out of the scope for mhcnuggets or it requires some additional tuning?
@RachelKarchin
Copy link
Collaborator

RachelKarchin commented Dec 4, 2020 via email

@vladimirkovacevic
Copy link
Author

Thank you @RachelKarchin. Looking forward to hearing from you.

@vladimirkovacevic
Copy link
Author

@RachelKarchin do you maybe have any update on this? Thank you.

@RachelKarchin
Copy link
Collaborator

RachelKarchin commented Jan 15, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@RachelKarchin @vladimirkovacevic