[ENH] `deepDNAshape` reworked

[fkiraly]

For DeepAptamer, one of the required components is the feature extractor (deep)DNAshape.

As per discussion in #82 (comment) and following, a package exists
https://github.com/JinsenLi/deepDNAshape

however it is not directly usable for us:

• it is not a pypi installable package
• it is tensorflow, not torch based
• it requires a comparably ancient numpy<1.24

In order to enable the reimplementation of DeepAptamer, we have hence discussed that the only feasible way forward (with an algorithm that includes deepDNAshape) is forking the deepDNAshape package into our library.

In the process of this, translating to torch to remove the version conflicts.

Crediting and license is straightforward:

• license is BSD-3, permissive. We need to credit code that is copied or inspired by
• in docstrings as well, we whould mention the original author, @JinsenLi to give clear credit, and cite the original publication

[satvshr]

For anyone working on this PR, #98 is the PR where DeepAptamer is being reimplimented, this is what I am using from the deepDNAshape library, so we do not need to reimpliment the entire library from scratch (though this simple function call seems to be stringed across 2/3 classes, at first glance).

Edit: @fkiraly also suggested using onnx as a translation tool to convert the tf weights to torch (though given it uses an older version (not sure about it) I am not sure if it will work).

[satvshr]

@fkiraly I also think we can involve Rohs lab(if they are interested) in this if they are interested, given they are huge (and maybe only) contributors in the whole "DNA shape" space.

[fkiraly]

I see - they are distinct from the deepaptamer authors, right?

[satvshr]

I see - they are distinct from the deepaptamer authors, right?

yes

[avinab-neogy]

Hi @satvshr @fkiraly ,
I am working on this issue now.
I went through #82 and have an understanding of how the deepDNAshape does not use the pentamer sliding window to calculate the MGW, Propeller twist, Helical twist and Roll so it returns the entire 138 features unlike the DNAshape library.

I also went through #98 and it is parked till this PR is merged right so should i go through that code?
thanks for the suggestion of onnx seems quite useful.

Will be updating my progress here.

[satvshr]

does not use the pentamer sliding window to calculate the MGW, Propeller twist, Helical twist and Roll so it returns the entire 138 features unlike the DNAshape library.

does it not? I thought the layer argument is the sliding window?

I also went through #98 and it is parked till this PR is merged right so should i go through that code?

As mentioned above, I would suggest looking at the predictor call only given that is all that we need.

[avinab-neogy]

does it not? I thought the layer argument is the sliding window?

You are right actually it still uses a sliding window but a variable one

deepDNAshape --seq AGAGATACGATACGA --feature ProT --layer 7

This uses a 2bp flanking region on each side, resulting in a 15-base receptive field (7*2 + 1). ( the DNAshape uses a fixed 5-base receptive field)
At positions near the sequence ends (e.g., i=1), some flanking positions are missing. However, thanks to the deep neural network’s convolutional filters and padding, deepDNAshape can still generate predictions by learning from partial context.
Increasing the --layer value expands the receptive field, providing more context ( on the other end ) especially helpful near edges, allowing reliable feature calculations even when flanking bases are limited.

Thanks Satvik, I understand this method better now!

As mentioned above, I would suggest looking at the predictor call only given that is all that we need.

Thanks a lot for specifying!

[satvshr]

However, thanks to the deep neural network’s convolutional filters and padding, deepDNAshape can still generate predictions by learning from partial context.

I do not think we need to trace as to why DNAshape gives only 126, but if I had to argue against the points youre trying to make, doesnt DNAshape also use neural networks?

Thanks Satvik, I understand this method better now!

Not a problem :D

[avinab-neogy]

I do not think we need to trace as to why DNAshape gives only 126, but if I had to argue against the points youre trying to make, doesnt DNAshape also use neural networks?

Nah it does not actually also I am not trying to argue :D . As I see it, DNAshape:

• Uses lookup tables of precomputed values (MGW, ProT, HelT, Roll, etc.) for each of the 1,024 possible 5-base pentamers ($4^5$ combinations) based on Monte Carlo simulations.

• Slides a fixed 5-base window along the sequence and queries those tables for each central position.

• Reports NA at the ends (where a full pentamer isn’t available) and removes those positions from the output.

Also I think it is important to trace why DNAshape gives 126 if we are trying to implement it!

On the contrary, deepDNAshape uses a CNN which is a deep neural network.

[satvshr]

Also I am not trying to argue :D

I meant it figuratively! As in "it's not worth debating that as it doesn't meet the end goal of the issue, and the issue can be solved without this question being answered". I probably could've phrased it better, apologies 😅