A Drone plugin for caching current workspace files between builds to reduce your build times. drone-cache
is a small CLI program, written in Go without any external OS dependencies (such as tar, etc).
With drone-cache
, you can provide your own cache key templates, specify archive format (tar, tar.gz, etc) and you can use an S3 bucket, Azure Storage, Google Cloud Storage or a mounted volume as storage for your cached files, even better you can implement your own storage backend to cover your use case.
For detailed usage information and a list of available options please take a look at usage and examples. If you want to learn more about custom cache keys, see cache key templates.
If you want to learn more about the story behind drone-cache
, you can read our blogpost Making Drone Builds 10 Times Faster!!
drone-cache
stores mounted directories and files under a key at the specified backend (by default S3).
Use this plugin to cache data that makes your builds faster. In the case of a cache miss or zero cache restore it will fail silently in won't break your running pipeline.
The best example would be to use this with your package managers such as Mix, Bundler or Maven. After your initial download, you can build a cache and then you can restore that cache in your next build.
With restored dependencies from a cache, commands like mix deps.get
will only need to download new dependencies, rather than re-download every package on each and every build.
The following .drone.yml
configuration show the most common use of drone-cache.
Note: These configs use drone 1.0 syntax. If you are using drone 0.8, check the examples in docs/examples/drone-0.8.md.
kind: pipeline
name: default
steps:
- name: restore-cache
image: meltwater/drone-cache
environment:
AWS_ACCESS_KEY_ID:
from_secret: aws_access_key_id
AWS_SECRET_ACCESS_KEY:
from_secret: aws_secret_access_key
pull: true
settings:
restore: true
cache_key: {{ .Commit.Branch }}-{{ checksum "go.mod" }} # default if ommitted is {{ .Commit.Branch }}
bucket: drone-cache-bucket
region: eu-west-1
mount:
- 'vendor'
- name: build
image: golang:1.13-alpine
pull: true
commands:
- apk add --update make git
- make drone-cache
- name: rebuild-cache
image: meltwater/drone-cache
pull: true
environment:
AWS_ACCESS_KEY_ID:
from_secret: aws_access_key_id
AWS_SECRET_ACCESS_KEY:
from_secret: aws_secret_access_key
settings:
rebuild: true
cache_key: {{ .Commit.Branch }}-{{ checksum "go.mod" }} # default if ommitted is {{ .Commit.Branch }}
bucket: drone-cache-bucket
region: eu-west-1
mount:
- 'vendor'
- examples for Drone 0.8, see docs/examples/drone-0.8.md
- examples for Drone 1.0, see docs/examples/drone-1.0.md
NAME:
Drone cache plugin - Drone cache plugin
USAGE:
drone-cache [global options] command [command options] [arguments...]
VERSION:
v1.0.4-18-g99c5e76-dirty
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--log.level value, --ll value log filtering level. ('error', 'warn', 'info', 'debug') (default: "info") [$PLUGIN_LOG_LEVEL, $ LOG_LEVEL]
--log.format value, --lf value log format to use. ('logfmt', 'json') (default: "logfmt") [$PLUGIN_LOG_FORMAT, $ LOG_FORMAT]
--repo.fullname value, --rf value repository full name [$DRONE_REPO]
--repo.namespace value, --rns value repository namespace [$DRONE_REPO_NAMESPACE]
--repo.owner value, --ro value repository owner (for Drone version < 1.0) [$DRONE_REPO_OWNER]
--repo.name value, --rn value repository name [$DRONE_REPO_NAME]
--repo.link value, --rl value repository link [$DRONE_REPO_LINK]
--repo.avatar value, --ra value repository avatar [$DRONE_REPO_AVATAR]
--repo.branch value, --rb value repository default branch [$DRONE_REPO_BRANCH]
--repo.private, --rp repository is private [$DRONE_REPO_PRIVATE]
--repo.trusted, --rt repository is trusted [$DRONE_REPO_TRUSTED]
--remote.url value, --remu value git remote url [$DRONE_REMOTE_URL]
--commit.sha value, --cs value git commit sha [$DRONE_COMMIT_SHA]
--commit.ref value, --cr value git commit ref (default: "refs/heads/master") [$DRONE_COMMIT_REF]
--commit.branch value, --cb value git commit branch (default: "master") [$DRONE_COMMIT_BRANCH]
--commit.message value, --cm value git commit message [$DRONE_COMMIT_MESSAGE]
--commit.link value, --cl value git commit link [$DRONE_COMMIT_LINK]
--commit.author.name value, --an value git author name [$DRONE_COMMIT_AUTHOR]
--commit.author.email value, --ae value git author email [$DRONE_COMMIT_AUTHOR_EMAIL]
--commit.author.avatar value, --aa value git author avatar [$DRONE_COMMIT_AUTHOR_AVATAR]
--build.event value, --be value build event (default: "push") [$DRONE_BUILD_EVENT]
--build.number value, --bn value build number (default: 0) [$DRONE_BUILD_NUMBER]
--build.created value, --bc value build created (default: 0) [$DRONE_BUILD_CREATED]
--build.started value, --bs value build started (default: 0) [$DRONE_BUILD_STARTED]
--build.finished value, --bf value build finished (default: 0) [$DRONE_BUILD_FINISHED]
--build.status value, --bstat value build status (default: "success") [$DRONE_BUILD_STATUS]
--build.link value, --bl value build link [$DRONE_BUILD_LINK]
--build.deploy value, --db value build deployment target [$DRONE_DEPLOY_TO]
--yaml.verified, --yv build yaml is verified [$DRONE_YAML_VERIFIED]
--yaml.signed, --ys build yaml is signed [$DRONE_YAML_SIGNED]
--prev.build.number value, --pbn value previous build number (default: 0) [$DRONE_PREV_BUILD_NUMBER]
--prev.build.status value, --pbst value previous build status [$DRONE_PREV_BUILD_STATUS]
--prev.commit.sha value, --pcs value previous build sha [$DRONE_PREV_COMMIT_SHA]
--backend value, -b value cache backend to use in plugin (s3, filesystem) (default: "s3") [$PLUGIN_BACKEND]
--mount value, -m value cache directories, an array of folders to cache [$PLUGIN_MOUNT]
--rebuild, --reb rebuild the cache directories [$PLUGIN_REBUILD]
--restore, --res restore the cache directories [$PLUGIN_RESTORE]
--cache-key value, --chk value cache key to use for the cache directories [$PLUGIN_CACHE_KEY]
--archive-format value, --arcfmt value archive format to use to store the cache directories (tar, gzip) (default: "tar") [$PLUGIN_ARCHIVE_FORMAT]
--compression-level value, --cpl value compression level to use for gzip compression when archive-format specified as gzip
(check https://godoc.org/compress/flate#pkg-constants for available options) (default: -1) [$PLUGIN_COMPRESSION_LEVEL]
--skip-symlinks, --ss skip symbolic links in archive [$PLUGIN_SKIP_SYMLINKS, $ SKIP_SYMLINKS]
--debug, -d debug [$PLUGIN_DEBUG, $ DEBUG]
--filesystem-cache-root value, --fcr value local filesystem root directory for the filesystem cache (default: "/tmp/cache") [$PLUGIN_FILESYSTEM_CACHE_ROOT, $ FILESYSTEM_CACHE_ROOT]
--endpoint value, -e value endpoint for the s3/cloud storage connection [$PLUGIN_ENDPOINT, $S3_ENDPOINT, $CLOUD_STORAGE_ENDPOINT]
--access-key value, --akey value AWS access key [$PLUGIN_ACCESS_KEY, $AWS_ACCESS_KEY_ID, $CACHE_AWS_ACCESS_KEY_ID]
--secret-key value, --skey value AWS/GCP secret key [$PLUGIN_SECRET_KEY, $AWS_SECRET_ACCESS_KEY, $CACHE_AWS_SECRET_ACCESS_KEY, $GCP_API_KEY]
--bucket value, --bckt value AWS bucket name [$PLUGIN_BUCKET, $S3_BUCKET, $CLOUD_STORAGE_BUCKET]
--region value, --reg value AWS bucket region. (us-east-1, eu-west-1, ...) [$PLUGIN_REGION, $S3_REGION]
--path-style, --ps use path style for bucket paths. (true for minio, false for aws) [$PLUGIN_PATH_STYLE]
--acl value upload files with acl (private, public-read, ...) (default: "private") [$PLUGIN_ACL]
--encryption value, --enc value server-side encryption algorithm, defaults to none. (AES256, aws:kms) [$PLUGIN_ENCRYPTION]
--azure-account-name value Azure Blob Storage Account Name [$PLUGIN_ACCOUNT_NAME, $AZURE_ACCOUNT_NAME]
--azure-account-key value Azure Blob Storage Account Key [$PLUGIN_ACCOUNT_KEY, $AZURE_ACCOUNT_KEY]
--azure-container-name value Azure Blob Storage container name [$PLUGIN_CONTAINER, $AZURE_CONTAINER_NAME]
--azure-blob-storage-url value Azure Blob Storage URL (default: "blob.core.windows.net") [$AZURE_BLOB_STORAGE_URL]
--sftp-cache-root value sftp root directory [$SFTP_CACHE_ROOT]
--sftp-username value sftp username [$SFTP_USERNAME]
--sftp-password value sftp password [$SFTP_PASSWORD]
--ftp-public-key-file value sftp public key file path [$SFTP_PUBLIC_KEY_FILE]
--sftp-auth-method value sftp auth method, defaults to none. (PASSWORD, PUBLIC_KEY_FILE) [$SFTP_AUTH_METHOD]
--sftp-host value sftp host [$SFTP_HOST]
--sftp-port value sftp port [$SFTP_PORT]
--help, -h show help
--version, -v print the version
$ docker run --rm \
-v "$(pwd)":/app \
-e DRONE_REPO=octocat/hello-world \
-e DRONE_REPO_BRANCH=master \
-e DRONE_COMMIT_BRANCH=master \
-e PLUGIN_MOUNT=/app/node_modules \
-e PLUGIN_RESTORE=false \
-e PLUGIN_REBUILD=true \
-e PLUGIN_BUCKET=<bucket> \
-e AWS_ACCESS_KEY_ID=<token> \
-e AWS_SECRET_ACCESS_KEY=<secret> \
meltwater/drone-cache
$ ./scripts/setup_dev_environment.sh
$ make test
OR
$ docker-compose up -d
$ go test ./..
Build the binary with the following commands:
$ go build .
Build the docker image with the following commands:
$ make container
Release management handled by CI pipeline. When you create a tag on master
branch, CI handles the rest.
You can find released artifacts (binaries, code, archives) under releases.
You can find released images at DockerHub.
PLEASE DO NOT INTRODUCE BREAKING CHANGES
Keep in mind that users usually use the image tagged with
latest
in their pipeline, please make sure you do not interfere with their working workflow.
We use SemVer for versioning. For the versions available, see the tags on this repository.
- @dim - Thanks for original work!
- @kakkoyun
- @salimane
- @AdamGlazerMW - Special thanks to Adam for the amazing artwork!
Also see the list of all contributors.
- github.com/bsm/drone-s3-cache (original work)
- github.com/Drillster/drone-volume-cache
Please read CONTRIBUTING.md to understand how to submit pull requests to us, and also see our code of conduct.
We keep all ideas for new features and bug reports in github.com/meltwater/drone-cache/issues.
One bigger area of future investment is to build a couple of new storage backends for caching the workspace files.
This project is licensed under the Apache License 2.0.