Redundant datasets being loaded

[erbriones]

This log snippet shows multiple workers running the same task. Note that each worker is running on a separate port.

07/17/2014 06:29:50 yerba.workqueue:90[INFO] WORKQUEUE coge-main: The workflow 1868 is scheduling job /opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868'
2014/07/17 18:29:52.04 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:51667) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
2014/07/17 19:09:57.13 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:37615) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
2014/07/17 19:58:37.56 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:34189) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
2014/07/17 21:02:12.21 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:33819) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
2014/07/17 22:10:28.04 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:35633) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
This entry below show multiple workers working on the same task. Each worker connects on a seperate port.

2014/07/17 23:29:28.33 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:38539) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
2014/07/18 00:30:13.76 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:59706) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
2014/07/18 01:17:00.17 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:51691) busy on '/opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868''
2014/07/18 01:54:22.86 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:51691) sending back file-8b6583d2337d354fd783542ab701e0d1-1868 to /storage/coge/tmp/staging/lht/1868
2014/07/18 01:54:22.87 [68341] work_queue_python: wq: coge.iplantcollaborative.org (127.0.0.1:51691) sending back file-5205ac7aadc00cfaa101ddf54244c354-log.done to /storage/coge/tmp/staging/lht/1868/log.done
07/18/2014 01:54:22 yerba.workflow:279[INFO] WORKFLOW Create Genome: Added info to job /opt/apache/coge/scripts/load_genome.pl  -user_name lht -name "" -desc "only scaffolds above 20kbp" -version "1.1" -restricted 1 -source_name "University of Queensland" -organism_id "39853" -type_id "1" -staging_dir '/storage/coge/tmp/staging/lht/1868' -fasta_files 'SSPACE_trimmed_out.final.scaffolds.fasta.rename.abv20k' -irods_files '' -config coge.conf -result_dir '/storage/coge/tmp/results/lht/1868'