ERROR: the value for setting EXP-TARGETS-BED must be specified logs-sbatch/slurm-18641451.out: qc-target-reads-gatk.sh ERROR: BED DOES NOT EXIST

[ptoru001]

Good morming Igor,

I have 7 samples 5 normal and two tumor.

The bed file has already bee added and is the same one as I used last time.

-rwx------ 1 torunp01 hackerlab 11302942 Jul 7 12:12 TruSeq_Exome_TargetedRegions_v1.2.bed

Yet I am getting this error:

ogs-sbatch/slurm-18641451.out: get-set-setting.sh ERROR: the value for setting EXP-TARGETS-BED must be specified
logs-sbatch/slurm-18641451.out: qc-target-reads-gatk.sh ERROR: BED DOES NOT EXIST
logs-sbatch/slurm-18641451.out: get-set-setting.sh ERROR: the value for setting EXP-TARGETS-BED must be specified
logs-sbatch/slurm-18641451.out: bam-ra-rc-gatk.sh ERROR: BED DOES NOT EXIST
logs-sbatch/slurm-18641451.out: wes.sh ERROR: SEGMENT bam-ra-rc-gatk DID NOT FINISH
logs-sbatch/slurm-18641452.out: get-set-setting.sh ERROR: the value for setting EXP-TARGETS-BED must be specified

Is there another file I must add?

Thank you
Pedro

[igordot]

Are you sure you are adding the BED file to the correct project directory?

Also, please check the entire log file. Just the error line is usually not enough.

[yaacoo]

I get exactly the same error, however I did not add any bed file because I'm working with WGS.

line 95 at:
https://github.com/igordot/sns/blob/main/segments/qc-target-reads-gatk.sh#L95

throws an exit 1 if there is no bed file, meaning bed is not optional.
I added a line that creates a bed file of all chromosome coordinates if there is no bed file present:

awk 'OFS="\t"{print $1, "0", $2}' ${ref_fastq}.fai > ${proj_dir}/wgs.bed
Does this seem like a good solution?

What is the easiest way to jump back into a specific stage or the workflow without running alignment again?

[igordot]

Since it's the same workflow for WES and WGS, you can add a full-genome BED file (the whole genome is targeted). There is a genome.bed in the reference directory.

You can run the pipeline again and it will skip the steps that successfully finished.

[yaacoo]

Yes that's what I figured. Thanks!

[ptoru001]

Thanks Igor,

all samples ran fine after after i deleted and reuploaded the bed file provided by GTC...

Are any of the files that are generated through WES route also good to use for copy number in gistic?

if not, what would be a way to obtain these .seg files?

Thanks for your time Igor.

-pedro

[igordot]

There is a wes-pairs-cnv route for copy number analysis using Control-FREEC. It does not generate SEG files, but you may be able to generate them from other output that it generates.

[ptoru001]

Thanks Igor,

I manually populated the fields in the pairs.csv file like this:
ptm8,ptm_3_214887
ptm8,ptm_4_215138
ptm8,ptm_5_213291
ptm8,ptm_6_215107
ptm8,ptm_7_215865

then I dbl checked that the bed file is currently in the settings.txt as well as in the directory itself and i get this error:
wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718525.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718526.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718527.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718528.out: wes-pairs-snv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718529.out: wes-pairs-snv.sh ERROR: BAM DOES NOT EXIST
logs-sbatch/slurm-18718530.out: wes-pairs-snv.sh ERROR: BAM DOES NOT EXIST
logs-sbatch/slurm-18718532.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718533.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718534.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718535.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718536.out: wes-pairs-cnv.sh ERROR: WRONG NUMBER OF ARGUMENTS SUPPLIED
logs-sbatch/slurm-18718537.out: wes-pairs-cnv.sh ERROR: BAM DOES NOT EXIST
logs-sbatch/slurm-18718538.out: wes-pairs-cnv.sh ERROR: BAM DOES NOT EXIST
logs-sbatch/slurm-18718543.out: get-set-setting.sh ERROR: the value for setting EXP-PROBES-BED must be specified
logs-sbatch/slurm-18718543.out: cnvs-wes-freec.sh ERROR: BED DOES NOT EXIST
logs-sbatch/slurm-18718545.out: get-set-setting.sh ERROR: the value for setting EXP-PROBES-BED must be specified
logs-sbatch/slurm-18718545.out: cnvs-wes-freec.sh ERROR: BED DOES NOT EXIST

how come the pipeline is not running?

Thank you
Pedro

[igordot]

There is probably a problem with file formatting (when you saved the file).

As I mentioned previously, if you see any errors, please check the full log file. The error line alone is not sufficient.

[ptoru001]

thanks for your feedback...this is the error message from logs-sbatch

although ive deleted and readded the settings for hg19, deleted and added the bed file to the settings path.

I am still getting the same errors.

apologies for the trouble and thanks for your feedback.

========== ROUTE: wes-pairs-cnv ==========

• proj_dir: /gpfs/data/hackerlab/2022-08-24
• tumor sample: ptm8
• normal sample: ptm_5_213291
• code_dir: /gpfs/data/hackerlab/2022-08-24/sns

CMD: bash /gpfs/data/hackerlab/2022-08-24/sns/segments/cnvs-wes-freec.sh /gpfs/data/hackerlab/2022-08-24 ptm8 /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm8.dd.ra.rc.bam ptm_5_213291 /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm_5_213291.dd.ra.rc.bam

========== SEGMENT: cnvs-wes-freec ==========

get-set-setting.sh ERROR: the value for setting GENOME-DIR must be specified

cnvs-wes-freec.sh ERROR: GENOME DIR DOES NOT EXIST

Fri Sep 2 12:00:02 EDT 2022

[igordot]

I don't understand. You say you are getting the same error, but posting a new error.

[ptoru001]

Good Morning,

This is from printing errors from slurm...i get no bed file error and no genome directory found.

These are the errors from printing logs-sbatch:

get-set-setting.sh ERROR: the value for setting GENOME-DIR must be specified
cnvs-wes-freec.sh ERROR: GENOME DIR DOES NOT EXIST
get-set-setting.sh ERROR: the value for setting EXP-PROBES-BED must be specified
cnvs-wes-freec.sh ERROR: BED DOES NOT EXIST
The bed file has been deleted and added but still gives errors above.

[igordot]

I understand where the errors are from. I meant that this is not an error you reported earlier. You introduced a new one.

When you edited settings.txt, you may have introduced non-UNIX line endings. You can check for yourself with cat -A settings.txt. You should see just $, not ^M$, at the end of every line.

To fix, run:

module load dos2unix/7.4.0
dos2unix settings.txt

[ptoru001]

hello igor,

this is what the settings.txt looks like
GENOME-DIR|/gpfs/data/igorlab/ref/hg19
REF-FASTA|/gpfs/data/igorlab/ref/hg19/genome.fa
REF-BWA|/gpfs/data/igorlab/ref/hg19/BWA/genome.fa
REF-DICT|/gpfs/data/igorlab/ref/hg19/genome.dict
EXP-TARGETS-BED|/gpfs/data/hackerlab/2022_08/TruSeq_Exome_TargetedRegions_v1.2.bed
EXP-PROBES-BED|/gpfs/data/hackerlab/2022_08/TruSeq_Exome_TargetedRegions_v1.2.bed
REF-CHROMSIZES|/gpfs/data/igorlab/ref/hg19/chrom.sizes

this is the error i am getting:

• FREEC: /gpfs/data/igorlab/software/FREEC/FREEC-11.6/src/freec
• sample T : ptm8
• BAM T : /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm8.dd.ra.rc.bam
• sample N : ptm_7_215865
• BAM N : /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm_7_215865.dd.ra.rc.bam
• probes original: /gpfs/data/hackerlab/2022_08/TruSeq_Exome_TargetedRegions_v1.2.bed
• probes fixed: /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/probes.bed
• ratio original: /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/ptm8.dd.ra.rc.bam_ratio.txt
• ratio fixed: /gpfs/data/hackerlab/2022-08-24/CNV-FREEC/ptm8-ptm_7_215865.ratio.txt
• CNVs original: /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/ptm8.dd.ra.rc.bam_CNVs
• CNVs fixed: /gpfs/data/hackerlab/2022-08-24/CNV-FREEC/ptm8-ptm_7_215865.CNVs.txt
• BAFs original: /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/ptm8.dd.ra.rc.bam_BAF.txt
• BAFs fixed: /gpfs/data/hackerlab/2022-08-24/CNV-FREEC/ptm8-ptm_7_215865.BAFs.txt
• graph: /gpfs/data/hackerlab/2022-08-24/CNV-FREEC/ptm8-ptm_7_215865.png

CMD: /gpfs/data/igorlab/software/FREEC/FREEC-11.6/src/freec -conf /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/config.txt

Control-FREEC v11.6 : a method for automatic detection of copy number alterations, subclones and for accurate estimation of contamination and main ploidy using deep-sequencing data
Multi-threading mode using 4 threads
..consider the sample being male
..Breakpoint threshold for segmentation of copy number profiles is 1.2
..telocenromeric set to 50000
..FREEC is not going to adjust profiles for a possible contamination by normal cells
..Window = 0 was set
..Output directory: .
..Directory with files containing chromosome sequences: /gpfs/data/igorlab/ref/iGenomes/Homo_sapiens/UCSC/hg19/Sequence/Chromosomes/
..will use a threshold of 10 read(s) per SNP position to calculate beta allel frequency (BAF) values
..Sample file: /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm8.dd.ra.rc.bam
..Sample input format: BAM
..will use this instance of samtools: 'samtools' to read BAM files
..Control file: /gpfs/data/hackerlab/2022-08-24/BAM-GATK-RA-RC/ptm_7_215865.dd.ra.rc.bam
..Input format for the control file: BAM
FREEC will create a pileup to compute BAF profile!
...File with SNPs : /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/snps.bed
..forceGCcontentNormalization was set to 1: will use GC-content to normalize the read count data
..minimal expected GC-content (general parameter "minExpectedGC") was set to 0.35
..maximal expected GC-content (general parameter "maxExpectedGC") was set to 0.55
..Minimal CNA length (in windows) is 5
..File with chromosome lengths: /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/chrs.len.txt
..File /gpfs/data/hackerlab/2022-08-24/logs-cnvs-wes-freec/ptm8-ptm_7_215865/chrs.len.txt was read
..Using the default minimal mappability value of 0.85
..Mappability file/gpfs/data/igorlab/ref/hg19/FREEC/out100m2_hg19.gem be used: all low mappability positions will be discarded
..uniqueMatch = FALSE
..average ploidy set to 2
..break-point type set to 4
..noisyData set to 1
..minimal number of reads per window in the control sample is set to 25
..Control-FREEC will not look for subclones
Creating Pileup file to compute BAF profile...
..will increase flanking regions by 107 bp
[mpileup] 1 samples in 1 input files

the CNV-FREEC folder is empty

thank you

[igordot]

I don't understand. There is no error.