Skip to content

Various docker improvements#652

Merged
MattWellie merged 7 commits intomainfrom
just-docker-improvements
Mar 30, 2026
Merged

Various docker improvements#652
MattWellie merged 7 commits intomainfrom
just-docker-improvements

Conversation

@MattWellie
Copy link
Copy Markdown
Collaborator

@MattWellie MattWellie commented Mar 29, 2026

Fixes

  • Improvements to the docker build process

Proposed Changes

  • Removes the redundant second download of htslib, already bundled in the bcftools download. Instead, build bcftools, cd, then build tabix/bgzip to the same location
  • Removes a couple of build layers, there's no benefit as they're all intended to be ephemoral
  • Update the UV install command to install all extras. Not 100% sure this is required, as the CPG install doesn't use this installation command, but best to update this while it's been spotted, instead of working back to try and discover why dependencies aren't satisfied
  • Removes a couple of libraries which are standard in linux, so a manual install is no-op
  • Swaps ENV VERSION for ARG VERSION - this isn't really meaningful here as we're currently using the Dockerfile line and a regular expression to extract the version value to make a build tag. If we do use this in future, an ENV variable can be changed during the build process, an ARG variable can't.

Extension plans

A more meaningful/beneficial change would be to split the dockerfiles into Tools and Talos. There are no nextflow or Hail Batch stages which require both Talos code and CLI tools, and the vast majority of docker build time is rebuilding bcftools from scratch each time the cache is invalidated.

A skinny image, featuring bcftools and echtvar, would be used during the preparation steps. The Talos image would be used for the final workflow. Both can be maintained separately, i.e. the tools image may never need updating, and the Talos image would build quickly.

Currently being tested locally

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the Docker build process for Talos images by simplifying bcftools/htslib build steps, reducing redundant package installs, and switching the image version marker from ENV VERSION to ARG VERSION for CI tagging.

Changes:

  • Adjust bcftools/htslib build to avoid a redundant standalone htslib download and install tabix/bgzip via the bundled htslib.
  • Simplify package installation and echtvar download steps in Dockerfiles.
  • Update CI version extraction to parse ARG VERSION=... instead of ENV VERSION=....

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File Description
src/talos/cpg_internal_scripts/CPG_Dockerfile Updates build stages and dependency installation for the CPG internal Talos image.
Dockerfile Updates the general Talos image build (bcftools build + python deps via uv).
.github/workflows/get_version.py Updates version parsing logic to match ARG VERSION in Dockerfile(s).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown

@EddieLF EddieLF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@MattWellie MattWellie merged commit 47b8b1b into main Mar 30, 2026
8 checks passed
@MattWellie MattWellie deleted the just-docker-improvements branch March 30, 2026 01:21
MattWellie added a commit that referenced this pull request Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants