Skip to content
Easy and secure way to copy data between Docker volumes, even across data centers
Branch: master
Clone or download
Latest commit a189451 May 8, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin Remove the coloring when outputing DVSYNC_TOKEN Jul 24, 2018
etc Handle ngrok errors and clean up Jul 6, 2018
media Add readme Jul 8, 2018
CODE_OF_CONDUCT.md Create CODE_OF_CONDUCT.md May 8, 2019
CONTRIBUTING.md
Dockerfile.client Handle ngrok errors and clean up Jul 6, 2018
Dockerfile.server Handle ngrok errors and clean up Jul 6, 2018
README.md Add missing docker command Mar 13, 2019
docker-compose.yml Add readme Jul 8, 2018

README.md


dvsync


Easy and secure way to copy data between Docker volumes, even across VPCs or data centers.

dvsync is a set of two containers running OpenSSH, ngrok and rsync that automatically create a secure, encrypted channel between each other and enable easy way to migrate data stored in Docker volumes.

Client: dvsync-client Server: dvsync-client


Running

⚠️ To establish a secure channel, dvsync uses ngrok and you need to pass your NGROK_AUTHTOKEN which can be found in ngrok dashboard.

dvsync will synchronize contents of its /data directory, therefore whatever data you want to sync, should be mounted under it.

Here are example ways to run it using Docker CLI, Docker Compose / Swarm or Kubernetes. Note that you can mix those, making data migration much easier.

Docker CLI

  1. Start a server where you want to copy data from:
$ docker run --rm -e NGROK_AUTHTOKEN="$NGROK_AUTHTOKEN" \
  --mount source=MY_SOURCE_VOLUME,target=/data,readonly \
  quay.io/suda/dvsync-server
  1. Once the server started, look into the logs and copy the DVSYNC_TOKEN
  2. Start the client where you want tot copy the data to:
$ docker run --rm -e DVSYNC_TOKEN="$DVSYNC_TOKEN" \
  --mount source=MY_TARGET_VOLUME,target=/data \
  quay.io/suda/dvsync-client

Using local source/target

Alternatively, if you want to copy this data to your local machine, you can mount a host directory as well:

$ docker run --rm -e DVSYNC_TOKEN="$DVSYNC_TOKEN" \
  -v $PWD:/data \
  quay.io/suda/dvsync-client

This can also be done other way around, when you start the server locally if you need to copy local data into the data center.

Docker Compose / Swarm

  1. Start a server where you want to copy data from:
version: '3.6'
services:
  dvsync-server:
    image: 'quay.io/suda/dvsync-server'
    environment:
      NGROK_AUTHTOKEN: ${NGROK_AUTHTOKEN}
    volumes:
      - type: volume
        source: MY_SOURCE_VOLUME
        target: /data
        read_only: true
volumes:
  MY_SOURCE_VOLUME:
  1. Once the server started, look into the logs and copy the DVSYNC_TOKEN
  2. Start the client where you want tot copy the data to:
version: '3.6'
services:
  dvsync-server:
    image: 'quay.io/suda/dvsync-client'
    environment:
      DVSYNC_TOKEN: ${DVSYNC_TOKEN}
    volumes:
      - type: volume
        source: MY_TARGET_VOLUME
        target: /data
volumes:
  MY_SOURCE_VOLUME:

Kubernetes

  1. Start a server where you want to copy data from:
apiVersion: v1
kind: Pod
metadata:
  name: dvsync-server
spec:
  containers:
  - image: quay.io/suda/dvsync-server
    name: dvsync-server
    env:
    - name: NGROK_AUTHTOKEN
      value: "REPLACE WITH YOUR NGROK_AUTHTOKEN"
    volumeMounts:
    - mountPath: /data
      name: MY_SOURCE_VOLUME
  volumes:
  - name: MY_SOURCE_VOLUME
  1. Once the server started, look into the logs and copy the DVSYNC_TOKEN
  2. Start the client where you want tot copy the data to:
apiVersion: v1
kind: Pod
metadata:
  name: dvsync-client
spec:
  containers:
  - image: quay.io/suda/dvsync-client
    name: dvsync-client
    env:
    - name: DVSYNC_TOKEN
      value: "REPLACE WITH YOUR DVSYNC_TOKEN"
    volumeMounts:
    - mountPath: /data
      name: MY_TARGET_VOLUME
  volumes:
  - name: MY_TARGET_VOLUME

Contributing

All contributions (no matter if small) are always welcome.

To see how you can help and where to start see Contributing file.

You can’t perform that action at this time.