Marking, removing, or keeping duplicates

[gh-byounggreenwald]

Hello again --

I have been reading through the deepSomatic paper again and don't see any mention of marking or discarding duplicates. What is the recommended method for handling duplicates in deepSomatic? If we mark them is this annotation leveraged? If we drop them are we hamstringing the algorithm because it was trained with duplicates present and thus its learned to leverage them in certain ways via its CNN?

I would assume the same thing is done as in deepVariant where they are excluded (and make little difference) google/deepvariant#384 but wanted to confirm as the paper didn't mention best practices.

Thanks :)

[kishwarshafin]

@gh-byounggreenwald

If we mark them is this annotation leveraged?

No, duplicate reads are not processed specially for generating examples. They are discarded similar to DeepVariant.

If we drop them are we hamstringing the algorithm because it was trained with duplicates present and thus its learned to leverage them in certain ways via its CNN?

I can confirm that all training data has duplicates marked and/or removed.

We haven't done an extensive investigation on what would happen if you don't mark duplicates, I would expect a similar effect on what you see with DeepVariant.

[gh-byounggreenwald]

Thank you very much!