Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update for v0.7.x downloader plugin interface #1

Merged
merged 3 commits into from
Mar 2, 2020

Conversation

mlin
Copy link
Collaborator

@mlin mlin commented Feb 19, 2020

The upcoming v0.7.x branch of miniwdl has a breaking change to the downloader plugin interface, sorry about this. It just adds two arguments to the entry point to give it access to the effective configuration and the task/workflow logger.

The first commit adds these two arguments and leaves everything else alone.

The second, optional commit gets the s3parcp docker image from the miniwdl configuration loader (which would source it either from the environment variable MINIWDL__S3PARCP__DOCKER_IMAGE or a configuration file [s3parcp] docker_image=...). Also it suggests filling that in at runtime using a WDL input instead of the f-string template. This is more a matter of preference clearly, either way works.

@kislyuk
Copy link
Contributor

kislyuk commented Feb 25, 2020

LGTM, please merge when ready.

@mlin mlin marked this pull request as ready for review March 2, 2020 19:31
@mlin
Copy link
Collaborator Author

mlin commented Mar 2, 2020

Mostly as a note to self -- here's how I minimally tested this

echo "FROM ubuntu:19.10
RUN apt-get update && apt-get install -y ca-certificates
ADD https://github.com/chanzuckerberg/s3parcp/releases/download/v0.1.2-alpha/s3parcp_0.1.2-alpha_Linux_x86_64.tar.gz /s3parcp/s3parcp.tar.gz
RUN tar zvx -C /s3parcp -f /s3parcp/s3parcp.tar.gz
RUN cp /s3parcp/s3parcp /usr/local/bin" | docker build -t s3parcp -

echo -e 'version 1.0\ntask cat { input { File in } command { cat ~{in} } output { String out=read_string(stdout()) } }' > /tmp/cat.wdl
MINIWDL__S3PARCP__DOCKER_IMAGE=s3parcp:latest miniwdl run /tmp/cat.wdl in=s3://mlin-west/alyssa_ben.txt --dir=/tmp --verbose

@kislyuk kislyuk merged commit 1f6bb30 into chanzuckerberg:master Mar 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants