Continuous Integration Via Github Actions (Overview)

[jjacobson95]

Full list of steps needed for CI.

• Build/Push DepMap to Dockerhub
• Build/Push CPTAC to Dockerhub
• Build/Push HCMI to Dockerhub
• Build/Push BeatAML to Dockerhub
• Build/Push MPNST to Dockerhub
• Build DepMap Samples File.
• Build CPTAC Samples File.
• Build HCMI Samples File.
• Build BeatAML Samples File.
• Build MPNST Samples File.
• Build BeatAML Drug Data
• Build DepMap Drug Data
• Check Drug Schema
• Build HCMI Data
• Build BeatAML Data
• Build DepMap Data
• Build CPTAC Data
• Build MPNST Data
• HCMI - check schema
• BeatAML - check schema
• DepMap - check schema
• CPTAC - check schema
• MPNST - check schema
• Upload Data to Figshare
• Data Visualization / Stats
• Update stats file with Visualization info and Figshare link.
• Update Docs with Data Visualization and Figshare Link
• Unit Test: Download (bash)
• Unit Test: Download (python)
• Unit Test: Load (python)
• Unit Test: Reformat (python)
• Unit Test: Join (python)
• Unit Test: Info (python)
• Push updated version of package to PyPI.

Build Steps:

• Create Dockerfile / Build Docker image
• Workflow update
• For samples file generation, create samples file. Save as artifact, pass to next samples file generator.

Update Steps:

• Checkout Action to get local repo files.
• Modify File via a python script.
• Push to Github.

Data Visualization Steps:

• Pull newly generated files via artifacts
• Generate Stats
• Generate Visualization PNGs
• Return all as artifacts. These will then be used in next steps.

Notes:

• Build steps will be sequential unless we decide to generate each samples file first.
• A stats file should be created. Maybe Yaml or Json. This will include info on stats and the Figshare link. This information will be parsed and docs will be updated with that info.

[jjacobson95]

A couple notes - I am checking off boxes that are completed in their own branches.
BeatAML/HCMI build steps and FigShare upload are in builder_branch_JJ.
Unit tests and PyPi upload is in package_dev_JJ.

To do:
Data visualization and the docs updates will be in the builder_branch_JJ.

[sgosline]

I'm working on the build scripts for cptac and cell_line data.

[jjacobson95]

Edited tasks above to include steps for dockerhub, drugs, schema.

[jjacobson95]

This is out of date. We are no longer using github actions to build all of our data with each commit.

Closing