Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use basespace CLI to download run data #13

Merged
merged 11 commits into from
Nov 4, 2021

Conversation

arisp99
Copy link
Member

@arisp99 arisp99 commented Nov 1, 2021

This PR replaces the old python script for downloading data from the Illumina BaseSpace Sequence Hub.

Resolves #8

@arisp99
Copy link
Member Author

arisp99 commented Nov 1, 2021

The old method of downloading data created a directory within the run directory, which is different than this current method. We need to ensure that we can still demux the data. This may involve changing the demux scripts with the updated directory structure. The contents of the directories are the same.

New download app:

$ tree new_method -L 1
new_method
├── <run_id>.json
├── Config
├── Data
├── InstrumentAnalyticsLogs
├── InterOp
├── Logs
├── Recipe
├── RTAComplete.txt
├── RTAConfiguration.xml
├── RTALogs
├── RTARead1Complete.txt
├── RTARead2Complete.txt
├── RTARead3Complete.txt
├── RTARead4Complete.txt
├── RunCompletionStatus.xml
├── RunInfo.xml
├── RunParameters.xml
└── SampleSheet.csv

7 directories, 11 files

Old download app:

$ tree old_method -L 2
old_method
├──<run_id>
│   ├── Config
│   ├── Data
│   ├── InstrumentAnalyticsLogs
│   ├── InterOp
│   ├── Logs
│   ├── Recipe
│   ├── RTAComplete.txt
│   ├── RTAConfiguration.xml
│   ├── RTALogs
│   ├── RTARead1Complete.txt
│   ├── RTARead2Complete.txt
│   ├── RTARead3Complete.txt
│   ├── RTARead4Complete.txt
│   ├── RunCompletionStatus.xml
│   ├── RunInfo.xml
│   ├── RunParameters.xml
│   └── SampleSheet.csv
└── nohup.out

8 directories, 11 files

@arisp99
Copy link
Member Author

arisp99 commented Nov 2, 2021

After some more investigation, this does not cause any issues with the demux app. Now instead of running the following:

singularity run \
  -B $resource_dir:/opt/resources -B $run_dir:/opt/analysis -B $bcl_dir:/opt/data \
  miptools.sif --app demux -s $sample_list -p $platform

where $run_dir is the directory of the project and bcl_dir=$run_dir"_<run id>", we must change $bcl_dir. We can just replace it with $run_dir as this is where the files from BaseSpace are now held:

singularity run \
  -B $resource_dir:/opt/resources -B $run_dir:/opt/analysis -B $run_dir:/opt/data \
  miptools.sif --app demux -s $sample_list -p $platform

We will get the same output. We may additionally want to consider slightly changing the demux script so that we no longer need to bind $run_dir to both /opt/analysis and /opt/data.

@arisp99 arisp99 added this to the 2.0.0 milestone Nov 3, 2021
@AshlinHarris AshlinHarris merged commit 7d4ecdb into bailey-lab:master Nov 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Downloading data from BaseSpace
2 participants