make duplicate handling configurable

[sreichl]

• Samtools & Samblaster
  • https://github.com/epigen/atacseq_pipeline/blob/main/workflow/rules/processing.smk
  • samtools filtering parameter -F 2316 configurable see docs & SAM flags
  • and/or samblaster duplicate removal configurable

  # Adding `remove_duplicates` to the `params` section:
  ...
  remove_duplicates = "--removeDups",
  ...
  
  # Including `{params.remove_duplicates}` in the `samblaster` command:
  ...
  samblaster {params.add_mate_tags} {params.remove_duplicates} 2> "{output.samblaster_log}" | \
  ....

• Macs2
  • https://github.com/epigen/atacseq_pipeline/tree/macs2_keep-dup_configurable
  • https://manpages.ubuntu.com/manpages/mantic/man1/macs2_callpeak.1.html
  • https://pypi.org/project/MACS2/
  • Removing Redundancy in ChIP-seq
• provide arguments and links for/against keeping duplicates and how to deal with them. show where to change if necessary in config

regarding more regions: I checked the alignment step (https://github.com/epigen/atacseq_pipeline/blob/main/workflow/rules/processing.smk) and as you suspect we flag the duplicates but don't remove them. I can't recall a specific argument for/agains it. Therefore, I can not claim this was intentional, on the other hand, most of the commands are quite specific and intentional. As far as I understand and have read up a little there are pro and cons associated with duplicate removal (pro: technical bias removed, con: potential biology removed; the usual tradeoff). My hope is that signals still remain the same, especially with many samples, filtering and normalization.

MACS2 automatically removes duplicates (with some threshold for max duplication). The duplicates macs2 removes is consistent with those marked by samblaster.

[sreichl]

FZ

# specify if duplicates should be kept during filtering bam files to define samtools view filtering flags
# filtered reads used as input for macs2 peak calling and counts
# warning: inclusion of duplicates should be intentional, and may lead to a large number of consensus regions