Merge pull request #615 from deeptools/develop

Develop

docs/content/News.rst

@@ -1,6 +1,14 @@
 News and Developments
 =====================
 
+Release 3.5.3
+-------------
+**14 October 2020**
+
+- Bug fix release: 
+   - Reads from scaffolds without any restriction enzym cut site are considered as 'same fragment'. An appearance of such a read will not lead to a crash anymore
+- Minor documentation improvements
+
 
 Release 3.5.2
 -------------

docs/content/list-of-tools.rst

@@ -83,8 +83,8 @@ Capture Hi-C analysis
     tools/chicDifferentialTest
     tools/chicPlotViewpoint
 
-.. contents::
-    :local:
+
+For single-cell Hi-C data analysis please use `scHiCExplorer <https://schicexplorer.readthedocs.io/>`__ .
 
 
 +--------------------------------------+------------------+-----------------------------------+---------------------------------------------+-----------------------------------------------------------------------------------+

docs/content/tools/chicViewpoint.rst

@@ -1,7 +1,7 @@
 .. _chicViewpoint:
 
 chicViewpoint
-===============
+=============
 
 .. argparse::
    :ref: hicexplorer.chicViewpoint.parse_arguments

docs/content/tools/hicBuildMatrix.rst

@@ -27,4 +27,11 @@ The lower resolutions need to be an integer multiplicative of the highest resolu
 Introducing with version 3.5 we support multiple restriction and dangling end sequences, and multiple restriction cut site files. 
 Hi-C protocols that use multiple restriction cut enzymes benefit from this and get now an improved QC report.
 Version 3.5 adds also the support for a chromosome size file which can help to get interaction matrices with a predefined size. Capture Hi-C or 
-single-cell Hi-C data, where it is not guaranteed that reads from all areas of the chromosome are present benefit from this latest improvement.
+single-cell Hi-C data, where it is not guaranteed that reads from all areas of the chromosome are present benefit from this latest improvement.
+
+
+Missing scaffolds or contigs in the Hi-C matrix
+-----------------------------------------------
+
+Restriction enzymes cut the DNA at their specific restriction cut site sequences. It can occur for scaffolds or contigs (less likely it happens for chromosomes) that it does not contain any of these cut sites.
+In this case, the reads from this scaffold or contig are considered as invalid and are part of the 'same restriction fragment'-statistics in the QC report.

docs/content/tools/hicDetectLoops.rst

@@ -1,7 +1,7 @@
 .. _hicDetectLoops:
 
 hicDetectLoops
-===============
+==============
 
 hicDetectLoops can detect enriched interaction regions (peaks / loops) based on a strict candidate selection, negative binomial distributions 
 and Wilcoxon rank-sum tests. 

docs/content/tools/hicTransform.rst

@@ -9,7 +9,7 @@ hicTransform
 
 
 Background
-===========
+----------
 
 hicTransform transforms a given input matrix into a new matrix using one of the following methods:
 

docs/content/tools/hicValidateLocations.rst

@@ -1,7 +1,7 @@
 .. _hicValidateLocations:
 
 hicValidateLocations
-=====================
+====================
 
 hicValidateLoops is a tool to compare the detect loops from hicDetectLoops (or from any other software as long as the data format is followed, see below) 
 with known peak protein locations to validate if the computed loops do have the expected anchor points. For example, loops in mammals are usually bound by CTCF or Cohesin, 
@@ -13,7 +13,7 @@ therefore it is important to know if the detect loops have protein peaks at thei
 
 
 Data format
-===========
+-----------
 
 The data format of hicDetectLoops output is:
 

hicexplorer/_version.py

@@ -2,4 +2,4 @@
 # This file is originally generated from Git information by running 'setup.py
 # version'. Distribution tarballs contain a pre-generated copy of this file.
 
-__version__ = '3.5.2'
+__version__ = '3.5.3'

hicexplorer/hicBuildMatrix.py

@@ -5,6 +5,7 @@
 from os import unlink
 import os
 from io import StringIO
+import traceback
 import warnings
 warnings.simplefilter(action="ignore", category=RuntimeWarning)
 warnings.simplefilter(action="ignore", category=PendingDeprecationWarning)
@@ -1002,8 +1003,9 @@ def process_data(pMateBuffer1, pMateBuffer2, pMinMappingQuality,
                                 len(restrictionSequence)
                             frag_end = max(mate1.pos + mate1.qlen, mate2.pos + mate2.qlen) - len(restrictionSequence)
                             mate_ref = pRefId2name[mate1.rname]
-                            has_rf.extend(sorted(
-                                pRfPositions[mate_ref][frag_start: frag_end]))
+                            if mate_ref in pRfPositions:
+                                has_rf.extend(sorted(
+                                    pRfPositions[mate_ref][frag_start: frag_end]))
 
                     # case when there is no restriction fragment site between the
                     # mates
@@ -1071,7 +1073,7 @@ def process_data(pMateBuffer1, pMateBuffer2, pMinMappingQuality,
                         mate_not_close_to_rf, count_inward, count_outward,
                         count_left, count_right, inter_chromosomal, short_range, long_range, pair_added, len(pMateBuffer1), pResultIndex, pCounter, out_bam_index_buffer]])
     except Exception as exp:
-        pQueueOut.put('Fail: ' + str(exp))
+        pQueueOut.put('Fail: ' + str(exp) + traceback.format_exc())
         return
     return
 
@@ -1147,7 +1149,7 @@ def main(args=None):
         rf_interval.extend(bed2interval_list(restrictionCutFile))
 
     rf_positions = intervalListToIntervalTree(rf_interval)
-
+    log.debug('rf_positions {}'.format(rf_positions.keys()))
     if args.binSize:
         bin_intervals = get_bins(args.binSize[0], chrom_sizes, args.region)
     else: