Run doublet detection on ground truth data

[sjspielman]

Purpose/implementation Section

Please link to the GitHub issue that this pull request addresses.

#446

What is the goal of this pull request?

This PR runs two (with a secret third!) doublet detection methods across four ground-truth datasets. The methods are scDblFinder (which runs a version of cxds for free) and scrublet. Results are saved in results/benchmark_results for further exploration in a subsequent PR.

Briefly describe the general approach you took to achieve this goal.

I wrote a couple different scripts to make this all happen, as documented in scripts/README.md.
First, I have a shell script to download the files from Zenodo, which also calls an R script to format into SCE and AnnData. My reasoning here was getting files into ScPCA-esque formats will make code more adaptable for future actual use.
Then, I have an R script that runs scDblFinder, and a python script that runs scrublet. At one point I thought about using reticulate, but then decided I'd rather not get into the R vs python environment weeds, and again keep things modular for each language for future portability.

There is also an overall script run_doublet-detection.sh to wrap everything.

If known, do you anticipate filing additional pull requests to complete this analysis module?

I sure do!

Results

What is the name of your results bucket on S3?

Everything is here: s3://researcher-654654257431-us-east-2/doublet-detection

What types of results does your code produce (e.g., table, figure)?

• SCE files with scDblFinder doublet results embedded
• TSV files with scrublet results

What is your summary of the results?

None yet - that's for the next PR where I analyze these outputs in a notebook.

Provide directions for reviewers

What are the software and computational requirements needed to be able to run the code in this PR?

• Can be run on a laptop.
• The module has both renv and conda environments, and directions for the latter are in README.md.

Are there particularly areas you'd like reviewers to have a close look at?

Have I gotten docs everywhere they need to be so far?

Is there anything that you want to discuss further?

Worth noting that the Dockerfile is not currently up-to-date with the environment, and currently considers only renv. I will need to get conda dependencies in here too, but waiting to do that until I take some time to evaluate a good base environment that could be used for this case.

Author checklists

Check all those that apply.
Note that you may find it easier to check off these items after the pull request is actually filed.

Analysis module and review

• This analysis module uses the analysis template and has the expected directory structure.
• The analysis module README.md has been updated to reflect code changes in this pull request.
• The analytical code is documented and contains comments.
• Any results and/or plots this code produces have been added to your S3 bucket for review.

Reproducibility checklist

• Code in this pull request has been added to the GitHub Action workflow that runs this module.
• The dependencies required to run the code in this pull request have been added to the analysis module Dockerfile.
• If applicable, the dependencies required to run the code in this pull request have been added to the analysis module conda environment.yml file.
• If applicable, R package dependencies required to run the code in this pull request have been added to the analysis module renv.lock file.

[jashapiro]

I gave some python style comments...

[allyhawkins]

Overall organization looks pretty good. My main comment is about making the decision to have these scripts run on any SCE or any AnnData, rather than hard coding in the filenames for benchmarking. I think I would add a few more options to the scripts, including the SCE file and an output file and then have the script run on a single sample. You already have a bash script for running the benchmarking datasets so you can loop through the script with the specific files you are interested in there. That will make it easier to re-use these scripts when we get to ScPCA data.

[sjspielman]

Code has now been heavily modularized and, I think, much improved; thanks for all the reviews! And bonus: the code now uses seeds 😄 Results are now all TSV files in my bucket.

I am hoping the code in here is "self-explanatory" from the docs 🤞 - if I'm very wrong, please stop reviewing and let me know!

[allyhawkins]

I think the reorg looks much better and a lot easier to follow! I just had a few remaining comments that I think should be addressed before merging.

[allyhawkins]

This file is specifically running scDblFinder so I might rename the script to be explicit about that.

[allyhawkins]

Same comment about the filename.

[allyhawkins]

Maybe a good idea to explicitly check that this file exists rather than the directory.

[jashapiro]

A few more little thoughts

[jashapiro]

You may want to set this as required=True rather than setting the default like this?

[sjspielman]

done all around where required (🥁 )

[allyhawkins]

A few small comments, but otherwise looks good!

[sjspielman]

@allyhawkins I ended up doing one more thing in 2b1410c - added a sample_var arg to the script and pass it along with the seed into the scdblfinder function, in case future us would like this.

[allyhawkins]

@allyhawkins I ended up doing one more thing in 2b1410c - added a sample_var arg to the script and pass it along with the seed into the scdblfinder function, in case future us would like this.

👍