Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gather step fails when sex-specific analysis is disabled for clustered endpoints #42

Open
Lipastomies opened this issue Oct 5, 2023 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@Lipastomies
Copy link
Contributor

In step2, the combination of 1) multiple clustered endpoints and 2) disabling sex-specific analysis causes the step2 pipeline to fail.

It works like this:

  1. when we disable sex-specific analysis, step2 step creates one sex-specific placeholder. When running clustered endpoints, there are instead n_endpoints sex-specific files.
if [[ "${run_sex_specific}" == "true" ]];
        then
for p in ${sep=" " phenolist}; do
# ...
done
# ...
fi
else
          ## sex specific analyses disabled but create the file so gather step is straightforward to implement
          touch ${prefix}"NOT_DONE.sex_spec.gz"
  1. when taking outputs for gathering step, we take the Array[Array[File]] of sex-specific files, of size N chunks * N clustered endpoints, transpose it, and then index it per endpoint. So indexing N clustered endpoints* N chunks array with each endpoint.
Array[Array[String]] pheno_results_sex = transpose(step2.sex_specifix)
scatter (pheno_result_idx in range(length(pheno_results))) {
        call gather {
            input: is_binary=is_binary, files=pheno_results[pheno_result_idx],
              sex_files = pheno_results_sex[pheno_result_idx], docker=docker
        }
        call summary {
            input: input_file=gather.pheweb, docker=docker
        }
    }
  1. The error is in the fact that when skipping sex-specific analysis, we create only N chunks*1 array of arrays, and the gather step requires N chunks * N endpoints, we get array indexing into nonexistent indices.

I'll fix this for R12 with an if-else in the gather step, replacing the sex-specific files with an empty array if no sex-specific analysis was done.

        Array[String] sex_files = if run_sex_specific then pheno_results_sex[pheno_result_idx] else []
        call gather {
            input: is_binary=is_binary, files=pheno_results[pheno_result_idx],
              sex_files = sex_files, docker=docker
        }
@Lipastomies Lipastomies added the bug Something isn't working label Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants