Using profile ./profiles/slurm for setting default command line arguments.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cluster nodes: 1000
Job stats:
job                        count
-----------------------  -------
all                            1
callable_bed                   1
create_cov_bed                 1
qc_admixture                   1
qc_plink                       1
qc_qc_plots                    1
qc_subsample_snps              1
qc_vcftools_individuals        1
total                          8

Select jobs to execute...

[Thu Feb  8 17:18:55 2024]
rule create_cov_bed:
    input: results/GCA_029891385.1/summary_stats/all_cov_sumstats.txt, results/GCA_029891385.1/callable_sites/all_samples.d4
    output: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    jobid: 1988
    benchmark: benchmarks/GCA_029891385.1/covbed/ERYs335_benchmark.txt
    reason: Missing output files: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=738803, mem_mib=704578, disk_mb=738803, disk_mib=704578, tmpdir=<TBD>

sbatch: error: Memory specification can not be satisfied
sbatch: error: Batch job submission failed: Requested node configuration is not available
Traceback (most recent call last):
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/./profiles/slurm/slurm-submit.py", line 59, in <module>
    print(slurm_utils.submit_job(jobscript, **sbatch_options))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 131, in submit_job
    raise e
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 129, in submit_job
    res = subprocess.check_output(["sbatch"] + optsbatch_options + [jobscript])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sbatch', '--partition=shared', '--time=4000', '--nodes=1', '--mem=738803', '--output=logs/slurm/slurm-%j.out', '--cpus-per-task=1', '/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.create_cov_bed.1988.sh']' returned non-zero exit status 1.
Error submitting jobscript (exit code 1):


[Thu Feb  8 17:18:58 2024]
rule qc_vcftools_individuals:
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    jobid: 2331
    reason: Missing output files: results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het, results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.imiss
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=16000, mem_mib=15259, disk_mb=440635, disk_mib=420223, tmpdir=<TBD>

Submitted job 2331 with external jobid '19017780'.
Trying to restart job 1988.
Select jobs to execute...

[Thu Feb  8 17:19:00 2024]
rule create_cov_bed:
    input: results/GCA_029891385.1/summary_stats/all_cov_sumstats.txt, results/GCA_029891385.1/callable_sites/all_samples.d4
    output: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    jobid: 1988
    benchmark: benchmarks/GCA_029891385.1/covbed/ERYs335_benchmark.txt
    reason: Missing output files: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=738803, mem_mib=704578, disk_mb=738803, disk_mib=704578, tmpdir=<TBD>

sbatch: error: Memory specification can not be satisfied
sbatch: error: Batch job submission failed: Requested node configuration is not available
Traceback (most recent call last):
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/./profiles/slurm/slurm-submit.py", line 59, in <module>
    print(slurm_utils.submit_job(jobscript, **sbatch_options))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 131, in submit_job
    raise e
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 129, in submit_job
    res = subprocess.check_output(["sbatch"] + optsbatch_options + [jobscript])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sbatch', '--partition=shared', '--time=4000', '--nodes=1', '--mem=738803', '--output=logs/slurm/slurm-%j.out', '--cpus-per-task=1', '/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.create_cov_bed.1988.sh']' returned non-zero exit status 1.
Error submitting jobscript (exit code 1):

Trying to restart job 1988.
Select jobs to execute...

[Thu Feb  8 17:19:01 2024]
rule create_cov_bed:
    input: results/GCA_029891385.1/summary_stats/all_cov_sumstats.txt, results/GCA_029891385.1/callable_sites/all_samples.d4
    output: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    jobid: 1988
    benchmark: benchmarks/GCA_029891385.1/covbed/ERYs335_benchmark.txt
    reason: Missing output files: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=738803, mem_mib=704578, disk_mb=738803, disk_mib=704578, tmpdir=<TBD>

sbatch: error: Memory specification can not be satisfied
sbatch: error: Batch job submission failed: Requested node configuration is not available
Traceback (most recent call last):
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/./profiles/slurm/slurm-submit.py", line 59, in <module>
    print(slurm_utils.submit_job(jobscript, **sbatch_options))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 131, in submit_job
    raise e
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 129, in submit_job
    res = subprocess.check_output(["sbatch"] + optsbatch_options + [jobscript])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sbatch', '--partition=shared', '--time=4000', '--nodes=1', '--mem=738803', '--output=logs/slurm/slurm-%j.out', '--cpus-per-task=1', '/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.create_cov_bed.1988.sh']' returned non-zero exit status 1.
Error submitting jobscript (exit code 1):

Trying to restart job 1988.
Select jobs to execute...

[Thu Feb  8 17:19:02 2024]
rule create_cov_bed:
    input: results/GCA_029891385.1/summary_stats/all_cov_sumstats.txt, results/GCA_029891385.1/callable_sites/all_samples.d4
    output: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    jobid: 1988
    benchmark: benchmarks/GCA_029891385.1/covbed/ERYs335_benchmark.txt
    reason: Missing output files: results/GCA_029891385.1/callable_sites/ERYs335_callable_sites_cov.bed
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=738803, mem_mib=704578, disk_mb=738803, disk_mib=704578, tmpdir=<TBD>

sbatch: error: Memory specification can not be satisfied
sbatch: error: Batch job submission failed: Requested node configuration is not available
Traceback (most recent call last):
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/./profiles/slurm/slurm-submit.py", line 59, in <module>
    print(slurm_utils.submit_job(jobscript, **sbatch_options))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 131, in submit_job
    raise e
  File "/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/profiles/slurm/slurm_utils.py", line 129, in submit_job
    res = subprocess.check_output(["sbatch"] + optsbatch_options + [jobscript])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/n/home08/dwhite/miniforge3/envs/snparcher/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['sbatch', '--partition=shared', '--time=4000', '--nodes=1', '--mem=738803', '--output=logs/slurm/slurm-%j.out', '--cpus-per-task=1', '/n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.create_cov_bed.1988.sh']' returned non-zero exit status 1.
Error submitting jobscript (exit code 1):

[Thu Feb  8 17:19:45 2024]
Error in rule qc_vcftools_individuals:
    jobid: 2331
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    conda-env: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/conda/efb07d19767c8764e9ef57f73e92b8b7_
    shell:
        
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --FILTER-summary --out results/GCA_029891385.1/QC/ERYs335
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --depth
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --het
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --missing-indv
        tail -n +2 results/GCA_029891385.1/QC/ERYs335.idepth | awk '$3>2 {print $1}'> results/GCA_029891385.1/QC/ERYs335.samps.txt
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: 19017780

Error executing rule qc_vcftools_individuals on cluster (jobid: 2331, external: 19017780, jobscript: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.qc_vcftools_individuals.2331.sh). For error details see the cluster log and the log files of the involved rule(s).
Trying to restart job 2331.
Select jobs to execute...

[Thu Feb  8 17:19:45 2024]
rule qc_vcftools_individuals:
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    jobid: 2331
    reason: Missing output files: results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het, results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.imiss
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=16000, mem_mib=15259, disk_mb=440635, disk_mib=420223, tmpdir=<TBD>

Submitted job 2331 with external jobid '19017799'.
[Thu Feb  8 17:20:16 2024]
Error in rule qc_vcftools_individuals:
    jobid: 2331
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    conda-env: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/conda/efb07d19767c8764e9ef57f73e92b8b7_
    shell:
        
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --FILTER-summary --out results/GCA_029891385.1/QC/ERYs335
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --depth
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --het
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --missing-indv
        tail -n +2 results/GCA_029891385.1/QC/ERYs335.idepth | awk '$3>2 {print $1}'> results/GCA_029891385.1/QC/ERYs335.samps.txt
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: 19017799

Error executing rule qc_vcftools_individuals on cluster (jobid: 2331, external: 19017799, jobscript: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.qc_vcftools_individuals.2331.sh). For error details see the cluster log and the log files of the involved rule(s).
Trying to restart job 2331.
Select jobs to execute...

[Thu Feb  8 17:20:16 2024]
rule qc_vcftools_individuals:
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    jobid: 2331
    reason: Missing output files: results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het, results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.imiss
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=16000, mem_mib=15259, disk_mb=440635, disk_mib=420223, tmpdir=<TBD>

Submitted job 2331 with external jobid '19017807'.
[Thu Feb  8 17:20:37 2024]
Error in rule qc_vcftools_individuals:
    jobid: 2331
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    conda-env: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/conda/efb07d19767c8764e9ef57f73e92b8b7_
    shell:
        
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --FILTER-summary --out results/GCA_029891385.1/QC/ERYs335
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --depth
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --het
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --missing-indv
        tail -n +2 results/GCA_029891385.1/QC/ERYs335.idepth | awk '$3>2 {print $1}'> results/GCA_029891385.1/QC/ERYs335.samps.txt
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: 19017807

Error executing rule qc_vcftools_individuals on cluster (jobid: 2331, external: 19017807, jobscript: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.qc_vcftools_individuals.2331.sh). For error details see the cluster log and the log files of the involved rule(s).
Trying to restart job 2331.
Select jobs to execute...

[Thu Feb  8 17:20:37 2024]
rule qc_vcftools_individuals:
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    jobid: 2331
    reason: Missing output files: results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het, results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.imiss
    wildcards: refGenome=GCA_029891385.1, prefix=ERYs335
    resources: mem_mb=16000, mem_mib=15259, disk_mb=440635, disk_mib=420223, tmpdir=<TBD>

Submitted job 2331 with external jobid '19017813'.
[Thu Feb  8 17:21:08 2024]
Error in rule qc_vcftools_individuals:
    jobid: 2331
    input: results/GCA_029891385.1/ERYs335_raw.vcf.gz
    output: results/GCA_029891385.1/QC/ERYs335.idepth, results/GCA_029891385.1/QC/ERYs335.imiss, results/GCA_029891385.1/QC/ERYs335.samps.txt, results/GCA_029891385.1/QC/ERYs335.FILTER.summary, results/GCA_029891385.1/QC/ERYs335.het
    conda-env: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/conda/efb07d19767c8764e9ef57f73e92b8b7_
    shell:
        
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --FILTER-summary --out results/GCA_029891385.1/QC/ERYs335
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --depth
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --het
        vcftools --gzvcf results/GCA_029891385.1/ERYs335_raw.vcf.gz --out results/GCA_029891385.1/QC/ERYs335 --missing-indv
        tail -n +2 results/GCA_029891385.1/QC/ERYs335.idepth | awk '$3>2 {print $1}'> results/GCA_029891385.1/QC/ERYs335.samps.txt
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: 19017813

Error executing rule qc_vcftools_individuals on cluster (jobid: 2331, external: 19017813, jobscript: /n/holyscratch01/davis_lab/dwhite/ERY/snpArcher/.snakemake/tmp.z1mxhwzl/snakejob.qc_vcftools_individuals.2331.sh). For error details see the cluster log and the log files of the involved rule(s).
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-02-08T165637.835606.snakemake.log