2021-10-29 12:23:24,130  INFO  - Running with database db.url = jdbc:hsqldb:mem:ba8b86f8-0d95-4c68-902d-640efc4c6fd5;shutdown=false;hsqldb.tx=mvcc
2021-10-29 12:23:36,948  INFO  - Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000
2021-10-29 12:23:36,972  INFO  - [RenameWorkflowOptionsInMetadata] 100%
2021-10-29 12:23:37,253  INFO  - Running with database db.url = jdbc:hsqldb:mem:7950d664-604e-4a3a-ae84-0d7bbdb86784;shutdown=false;hsqldb.tx=mvcc
2021-10-29 12:23:38,122  INFO  - Slf4jLogger started
2021-10-29 12:23:38,455 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO  - Workflow heartbeat configuration:
{
  "cromwellId" : "cromid-153e7ae",
  "heartbeatInterval" : "2 minutes",
  "ttl" : "10 minutes",
  "failureShutdownDuration" : "5 minutes",
  "writeBatchSize" : 10000,
  "writeThreshold" : 10000
}
2021-10-29 12:23:38,697 cromwell-system-akka.dispatchers.service-dispatcher-15 INFO  - Metadata summary refreshing every 1 second.
2021-10-29 12:23:38,726 cromwell-system-akka.dispatchers.service-dispatcher-15 INFO  - No metadata archiver defined in config
2021-10-29 12:23:38,726 cromwell-system-akka.dispatchers.service-dispatcher-15 INFO  - No metadata deleter defined in config
2021-10-29 12:23:38,921 cromwell-system-akka.actor.default-dispatcher-2 INFO  - KvWriteActor configured to flush with batch size 200 and process rate 5 seconds.
2021-10-29 12:23:38,925 cromwell-system-akka.dispatchers.service-dispatcher-17 INFO  - WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds.
2021-10-29 12:23:38,988  WARN  - 'docker.hash-lookup.gcr-api-queries-per-100-seconds' is being deprecated, use 'docker.hash-lookup.gcr.throttle' instead (see reference.conf)
2021-10-29 12:23:39,022 cromwell-system-akka.dispatchers.engine-dispatcher-9 INFO  - CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds.
2021-10-29 12:23:39,163 cromwell-system-akka.dispatchers.engine-dispatcher-30 INFO  - JobExecutionTokenDispenser - Distribution rate: 1 per 2 seconds.
2021-10-29 12:23:39,330 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO  - SingleWorkflowRunnerActor: Version 65
2021-10-29 12:23:39,350 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO  - SingleWorkflowRunnerActor: Submitting workflow
2021-10-29 12:23:39,556 cromwell-system-akka.dispatchers.api-dispatcher-31 INFO  - Unspecified type (Unspecified version) workflow 6c07851c-03c1-4d65-9936-bb0192c564c4 submitted
2021-10-29 12:23:39,657 cromwell-system-akka.dispatchers.engine-dispatcher-26 INFO  - SingleWorkflowRunnerActor: Workflow submitted UUID(6c07851c-03c1-4d65-9936-bb0192c564c4)
2021-10-29 12:23:39,665 cromwell-system-akka.dispatchers.engine-dispatcher-29 INFO  - 1 new workflows fetched by cromid-153e7ae: 6c07851c-03c1-4d65-9936-bb0192c564c4
2021-10-29 12:23:39,672 cromwell-system-akka.dispatchers.engine-dispatcher-26 INFO  - WorkflowManagerActor: Starting workflow UUID(6c07851c-03c1-4d65-9936-bb0192c564c4)
2021-10-29 12:23:39,675 cromwell-system-akka.dispatchers.engine-dispatcher-26 INFO  - WorkflowManagerActor: Successfully started WorkflowActor-6c07851c-03c1-4d65-9936-bb0192c564c4
2021-10-29 12:23:39,676 cromwell-system-akka.dispatchers.engine-dispatcher-26 INFO  - Retrieved 1 workflows from the WorkflowStoreActor
2021-10-29 12:23:39,743 cromwell-system-akka.dispatchers.engine-dispatcher-29 INFO  - WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes.
2021-10-29 12:23:40,080 cromwell-system-akka.dispatchers.engine-dispatcher-29 INFO  - MaterializeWorkflowDescriptorActor [UUID(6c07851c)]: Parsing workflow as WDL 1.0
2021-10-29 12:23:44,180 cromwell-system-akka.dispatchers.engine-dispatcher-26 INFO  - Not triggering log of token queue status. Effective log interval = None
2021-10-29 12:23:45,915 cromwell-system-akka.dispatchers.engine-dispatcher-29 INFO  - MaterializeWorkflowDescriptorActor [UUID(6c07851c)]: Call-to-Backend assignments: atac.macs2_signal_track -> slurm, atac.error_input_data -> slurm, atac.qc_report -> slurm, atac.call_peak_pooled -> slurm, atac.idr -> slurm, atac.call_peak_pr1 -> slurm, atac.align -> slurm, atac.pool_ta -> slurm, atac.idr_pr -> slurm, atac.compare_signal_to_roadmap -> slurm, atac.count_signal_track -> slurm, atac.bam2ta -> slurm, atac.xcor -> slurm, atac.overlap_ppr -> slurm, atac.overlap -> slurm, atac.reproducibility_overlap -> slurm, atac.fraglen_stat_pe -> slurm, atac.macs2_signal_track_pooled -> slurm, atac.overlap_pr -> slurm, atac.read_genome_tsv -> slurm, atac.call_peak_ppr1 -> slurm, atac.frac_mito -> slurm, atac.call_peak_ppr2 -> slurm, atac.tss_enrich -> slurm, atac.call_peak_pr2 -> slurm, atac.bam2ta_no_dedup -> slurm, atac.gc_bias -> slurm, atac.filter -> slurm, atac.jsd -> slurm, atac.count_signal_track_pooled -> slurm, atac.pool_ta_pr1 -> slurm, atac.spr -> slurm, atac.annot_enrich -> slurm, atac.pool_ta_pr2 -> slurm, atac.idr_ppr -> slurm, atac.call_peak -> slurm, atac.filter_no_dedup -> slurm, atac.reproducibility_idr -> slurm, atac.align_mito -> slurm, atac.pool_blacklist -> slurm, atac.preseq -> slurm
2021-10-29 12:23:46,434 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,440 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,446 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,452 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,453 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,454 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,455 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,455 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,456 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,457 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,458 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,459 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,459 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,460 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,460 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,461 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,461 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,462 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,464 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,465 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,465 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,465 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,466 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,466 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,467 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,477 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,478 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,479 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,479 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,479 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,479 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,481 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,481 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,482 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,483 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,483 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,484 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,485 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,485 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,486 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:46,487 cromwell-system-akka.dispatchers.backend-dispatcher-37 WARN  - slurm [UUID(6c07851c)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions.
2021-10-29 12:23:50,843 cromwell-system-akka.dispatchers.engine-dispatcher-28 INFO  - WorkflowExecutionActor-6c07851c-03c1-4d65-9936-bb0192c564c4 [UUID(6c07851c)]: Starting atac.read_genome_tsv
2021-10-29 12:23:51,185 cromwell-system-akka.dispatchers.engine-dispatcher-28 INFO  - Assigned new job execution tokens to the following groups: 6c07851c: 1
2021-10-29 12:23:53,268 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO  - 6c07851c-03c1-4d65-9936-bb0192c564c4-EngineJobExecutionActor-atac.read_genome_tsv:NA:1 [UUID(6c07851c)]: Could not copy a suitable cache hit for 6c07851c:atac.read_genome_tsv:-1:1. No copy attempts were made.
2021-10-29 12:23:53,315 cromwell-system-akka.dispatchers.backend-dispatcher-40 WARN  - BackgroundConfigAsyncJobExecutionActor [UUID(6c07851c)atac.read_genome_tsv:NA:1]: Unrecognized runtime attribute keys: disks
2021-10-29 12:23:53,432 cromwell-system-akka.dispatchers.backend-dispatcher-40 INFO  - BackgroundConfigAsyncJobExecutionActor [UUID(6c07851c)atac.read_genome_tsv:NA:1]: `echo "$(basename /cromwell-executions/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/inputs/-230992159/hg38.local.tsv)" > genome_name
# create empty files for all entries
touch ref_fa bowtie2_idx_tar chrsz gensz blacklist blacklist2
touch ref_mito_fa
touch bowtie2_mito_idx_tar
touch tss tss_enrich # for backward compatibility
touch dnase prom enh reg2map reg2map_bed roadmap_meta
touch mito_chr_name
touch regex_bfilt_peak_chr_name

python <<CODE
import os
with open('/cromwell-executions/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/inputs/-230992159/hg38.local.tsv','r') as fp:
    for line in fp:
        arr = line.strip('\n').split('\t')
        if arr:
            key, val = arr
            with open(key,'w') as fp2:
                fp2.write(val)
CODE`
2021-10-29 12:23:53,979 cromwell-system-akka.dispatchers.backend-dispatcher-40 INFO  - BackgroundConfigAsyncJobExecutionActor [UUID(6c07851c)atac.read_genome_tsv:NA:1]: executing: cat << EOF > /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/script.caper
#!/bin/bash

if [ 'true' == 'true' ] && [ 'conda' == 'singularity' ] || \
   [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'library://leepc12/default/atac-seq-pipeline:v2.0.0' ]
then
    mkdir -p $HOME/.singularity/lock/
    flock --exclusive --timeout 600 \
        $HOME/.singularity/lock/`echo -n 'library://leepc12/default/atac-seq-pipeline:v2.0.0' | md5sum | cut -d' ' -f1` \
        singularity exec --containall library://leepc12/default/atac-seq-pipeline:v2.0.0 echo 'Successfully pulled library://leepc12/default/atac-seq-pipeline:v2.0.0'

    singularity exec --cleanenv --home=/local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv \
        --bind=,/local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv:/cromwell-executions/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv \
         \
        library://leepc12/default/atac-seq-pipeline:v2.0.0 /bin/bash /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/script

elif [ 'true' == 'true' ] && [ 'conda' == 'conda' ] || \
     [ 'true' == 'false' ] && [ 'true' == 'true' ] && [ ! -z 'encode-atac-seq-pipeline' ]
then
    
shopt -s nullglob
sed -i 's#/cromwell-executions/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv#/local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv#g' /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/script `dirname /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/script`/write_*.tmp

    conda run --name=encode-atac-seq-pipeline /bin/bash /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/script

else
    /bin/bash /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/script
fi

EOF

for ITER in 1 2 3; do
    sbatch --export=ALL -J cromwell_6c07851c_read_genome_tsv -D /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv -o /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/stdout -e /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/stderr \
         --account mpsnyder \
        -n 3 --ntasks-per-node=1 --cpus-per-task=1 --mem=2048M --time=4  \
         \
        /local/scratch/asanghi/atac/6c07851c-03c1-4d65-9936-bb0192c564c4/call-read_genome_tsv/execution/script.caper && break
    sleep 30
done
2021-10-29 12:23:58,961 cromwell-system-akka.dispatchers.backend-dispatcher-40 INFO  - BackgroundConfigAsyncJobExecutionActor [UUID(6c07851c)atac.read_genome_tsv:NA:1]: job id: 35014
2021-10-29 12:23:58,965 cromwell-system-akka.dispatchers.backend-dispatcher-39 INFO  - BackgroundConfigAsyncJobExecutionActor [UUID(6c07851c)atac.read_genome_tsv:NA:1]: Status change from - to WaitingForReturnCode