Major performance improvement by exchange vectorized Numpy code with Numba #40
Conversation
| # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER | ||
| # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, | ||
| # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
| # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
There was a problem hiding this comment.
Something I'm curious about, in every single file you have a copy of the licence. What's the point? You already maintain a copy of the licence under conkit/LICENCE.txt and surely having a copy everywhere means that once a year you need to go through and change it everywhere.
There was a problem hiding this comment.
It's just in case, once the repo is installed via PyPi, the license is not copied across and so this way it's explicit on a repo and per-file basis.
conkit/core/ext/_contactmap.py
Outdated
| for i in range(X.shape[0]): | ||
| for j in numba.prange(i + 1, X.shape[0]): | ||
| for j in range(i + 1, X.shape[0]): |
There was a problem hiding this comment.
Is numba.prange not an advantage here?
There was a problem hiding this comment.
numba.prange assumes that values in each iteration can be computed independently of the previous. Since I'm skipping all throwables[j] that are already flagged for removal, vectorizing this loop might cause computations where they are not needed.
Added
numbaadded as dependencyChanged
SequenceFile.calculate_freqbackend changed fromnumpytonumbafor faster computationSequenceFile.calculate_weightsbackend changed fromnumpytonumbafor faster computationSequenceFile.filterbackend changed fromnumpytonumbafor faster computationSequenceFile.filter_gappedbackend changed fromnumpytonumbafor faster computationSequenceFile.calculate_weightsrenamed toSequenceFile.get_weightsSequenceFile.compute_freqrenamed toSequenceFile.get_frequencyContactMap.singletonsbackend changed fromnumpytonumbafor faster computationBandwidthbackend changed fromnumpytonumbafor faster computationContactMap.short_range_contactsrenamed toContactMap.short_rangeContactMap.medium_range_contactsrenamed toContactMap.medium_rangeContactMap.long_range_contactsrenamed toContactMap.long_rangeContactMap.calculate_scalar_scorerenamed toContactMap.set_scalar_scoreContactMap.calculate_contact_densityrenamed toContactMap.get_contact_densityContactMap.calculate_jaccard_indexrenamed toContactMap.get_jaccard_indexFixed
SequenceFile.filterto removeSequenceentries reliablyContactMapMatrixFigurewhengapvariable was less than 1