Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement TSO v2 ILMN pipeline orchestration - baseline #135

Closed
victorskl opened this issue Mar 5, 2024 · 2 comments · Fixed by #145 or #281
Closed

Implement TSO v2 ILMN pipeline orchestration - baseline #135

victorskl opened this issue Mar 5, 2024 · 2 comments · Fixed by #145 or #281
Assignees
Labels
epic feature New feature pipeline Workflow/Pipeline Manager

Comments

@victorskl
Copy link
Member

victorskl commented Mar 5, 2024

Context:

As the Phase 1 of the ICAv2 pipeline migration plan; we need to do:

Steps

1. Generate v2-SampleSheet in the Run data location in ICAv1 GDS
2. Copy the Run (raw) data from ICAv1 to (BSSH linked) ICAv2
3. Auto-launch BCL Convert (version 4.2.7) using ILMN provided bundle
4. Deploy bssh manager service (renamed to bssh_fastq_copy_service)
5. Deploy the icav2_copy_batch_utility as its own step functions service
6. Deploy the cttso_v2_pipeline_manager service

[EDIT]
Steps 1, 2, 3 as an "bridge" application break it down to another task #140

Prior work items:

There are already prior arts / existing tooling build for this by Alexis. We now basically need to operationalise these tools using StepFunction as its orchestration around executing the steps mentioned above^^.

@victorskl victorskl added the pipeline Workflow/Pipeline Manager label Mar 5, 2024
@victorskl victorskl changed the title Implement bssh manager application - baseline Implement bssh_fastq_copy_service application - baseline Mar 6, 2024
@victorskl
Copy link
Member Author

victorskl commented Mar 6, 2024

Renamed and rearranged the planning steps a bit.
See the following Slack discussions for the context that drive this story card.

@victorskl victorskl changed the title Implement bssh_fastq_copy_service application - baseline Implement TSO v2 ILMN pipeline orchestration - baseline Mar 6, 2024
@alexiswl alexiswl linked a pull request Mar 10, 2024 that will close this issue
7 tasks
@victorskl victorskl added epic feature New feature labels May 6, 2024
@victorskl
Copy link
Member Author

Service namespace: orcabus.cttsov2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic feature New feature pipeline Workflow/Pipeline Manager
Projects
None yet
2 participants