Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Pipeline for ChIP-seq preprocessing


Here is the current pipeline used for ChIP-seq preprocessing, which includes the following steps:

  • align the fastq data to reference genome by bowtie2.
  • run FastQC to check the sequencing quality.
  • remove all reads duplications of the aligned data.
  • generate TDF files for browsing in IGV.
  • run PhantomPeak to check the quality of ChIP.
  • run diffRepeats on multi- and un-mapped reads.
  • run ngs.plot to investigate the enrichment of ChIP-seq data at TSS, TES, and genebody (only implemented in local version, not lsf cluster computing).

The pipeline work flow is:

work flow


The softwares used in this pipeline are:

Install above softwares and make sure they are in $PATH.


Put the scripts in ./bin to a place in $PATH or add ./bin to $PATH.


Update the config.yaml file to set the configuration required for your project.

Then execute:

python config.yaml

Or on an LSF cluster:

nohup python config.yaml &

After the running of the pipeline, then to summarize the result:

python config.yaml

For the configuration yaml file, project_dir: ~/projects/test_ChIP-seq and data_dir: "data" mean the data folder is ~/projects/test_ChIP-seq/data, and the results will be put in the same folder. Fastq files should be under ~/projects/test_ChIP-seq/data/fastq folder. Now *.fastq, *.fq, *.gz (compressed fastq) files are acceptable. aligner currently only supports using bowtie2.

The location of,, and config.yaml doesn't matter at all. But I prefer to put them under project/script/preprocess folder.


  • The alignment step includes parsing of results into unique-mapped, multi-mapped, and un-mapped bam files. The unique-mapped results are sent to rmdup, while the multi- and un-mapped results are used to run diffRepeats. Settings to determine unique- and multi-mapped reads are in config.yaml.
  • To make ngs.plot part work, please name the fastq files in this way:
Say condition A, B, each with 2 replicates, and one DNA input per condition. 
Name the files as A_rep1.fastq, A_rep2.fastq, A_input.fastq, B_rep1.fastq, 
B_rep2.fastq, and B_input.fastq.The key point is to make the same condition
 samples with common letters and input samples contain "input" or "Input"
  • If use want to only run to some specific step, just modify the function name in pipeline_run in
  • If the data are pair-end, follow this step:
    • Modify the config.yaml file, change "pair_end" to "yes".
    • Modify the config.yaml file, change "input_files" to "*R1*.fastq.gz" or "*R1*.fastq".
    • Make sure the fastq files named as "*R1*" and "*R2*" pattern.
  • if you want to use cluster:
    • Edit '~/.bash_profile' to make sure all paths in $PATH.
    • Modify config.yaml to fit your demands.
    • multithread in determines the number of concurrent jobs to be submitted to cluster nodes by ruffus. A default value of 10 is used.


Bowtie2 allows multiple hits reads, and breaks the assumption of phantomPeak:

It is EXTREMELY important to filter out multi-mapping reads from the BAM/tagAlign
 files. Large number of multimapping reads can severly affect the phantom peak
 coefficient and peak calling results.

So be careful to interpret NSC and RSC in Bowtie2 alignment results.


In Bowtie2, default parameters are used.


A preprocessing pipeline for ChIP-seq, including alignment, quality control, and visualization.




No packages published