dotmesh (dm) is like git for your data volumes (databases, files etc) in Docker and Kubernetes
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
cmd potential fix for unclosed files Oct 22, 2018
docs NFC: Documented what to do when changing dependencies Oct 11, 2018
etcd-container-image Initial commit. Feb 15, 2018
kubernetes #311, #537: fixup image Aug 7, 2018
pkg FIX: Avoid discarding connection errors, by lazily connecting on demand. Oct 16, 2018
scripts NFC: Clearer logging of success/failure in the smoke tests, so it's e… Oct 11, 2018
tests gateway#49: tests for s3 api -> list files at snapshot and read file … Oct 17, 2018
vendor vendor updated Oct 2, 2018
zfs-userland-builder #440: only include the files that are needed, break docker cache to m… Aug 2, 2018
.dockerignore Initial commit. Feb 15, 2018
.gitignore updated gitignore to ignore vscode Sep 28, 2018
.gitlab-ci.yml NFC: switch to triggering e2e with ci_job_token Oct 2, 2018
BUILD.bazel NFC: updates to build files after master merge Jul 23, 2018
Gopkg.lock deps updated Oct 2, 2018
Gopkg.toml correct ignore Jul 25, 2018
ISSUE_TEMPLATE.md Create ISSUE_TEMPLATE.md Jul 23, 2018
LICENSE Initial commit. Feb 15, 2018
Makefile FIX: allow easier sync of changed files inside vagrant without having… Apr 27, 2018
README.md NFC: formatting Sep 5, 2018
Vagrantfile Initial commit. Feb 15, 2018
WORKSPACE #527: bump to bleeding edge of docker rules Jul 31, 2018
build-lib.sh FIX: targets Jul 31, 2018
datadot.png Initial commit. Feb 15, 2018
dev.sh FIX: allow easier sync of changed files inside vagrant without having… Apr 27, 2018
mark-cleanup.sh Initial commit. Feb 15, 2018
prep-tests.sh Initial commit. Feb 15, 2018
prime.sh #440: remove unnecessary prime.sh images Aug 2, 2018
rebuild.sh FIX: shift back to pushing using docker, bazel seems to occasionally … Jul 27, 2018
rebuild_client.sh FIX: shift back to pushing using docker, bazel seems to occasionally … Jul 27, 2018
rebuild_in_container_without_bazel.sh FIX: s/CI_DOCKER_SERVER_IMAGE/STABLE_CI_DOCKER_SERVER_IMAGE/ change h… Aug 10, 2018
rebuild_operator.sh FIX: shift back to pushing using docker, bazel seems to occasionally … Jul 27, 2018
rebuild_provisioner.sh FIX: shift back to pushing using docker, bazel seems to occasionally … Jul 27, 2018
rebuild_server.sh FIX: s/CI_DOCKER_SERVER_IMAGE/STABLE_CI_DOCKER_SERVER_IMAGE/ change h… Aug 10, 2018
rebuild_without_bazel.sh FIX: use SKIP_SERVER and ensure lowercase OS argument for local-stack… Aug 8, 2018
release-hosts NFC: Strict host key checking for `get.dotmesh.io` Aug 17, 2018
release-images.sh FIX: set up defaults before doing rebuild Jul 27, 2018
test.sh FIX: quotes needed for -n to function Aug 1, 2018
up.sh Initial commit. Feb 15, 2018
version_status.sh NFC: shift versioner run to setup-env so we see it in the output Jul 27, 2018

README.md

dotmesh: git for data

pipeline status

Dotmesh is a git-like CLI for capturing, organizing and sharing application states.

In other words, it's a snapshotting tool for databases and other filesystem states.

The application states that dotmesh captures are stored in datadots.

It can capture the state of multiple databases, each one in a subdot, in a single atomic commit.

installing on docker (Mac or Ubuntu 16.04+)

Install the dotmesh client dm:

sudo curl -sSL -o /usr/local/bin/dm \
    https://get.dotmesh.io/$(uname -s)/dm

Make the client binary executable.

sudo chmod +x /usr/local/bin/dm

Then use the client to install dotmesh-server, assuming you have Docker installed and your user account has access to the Docker daemon.

dm cluster init
Checking suitable Docker is installed... yes, got 17.12.0-ce.
Checking dotmesh isn't running... done.
Pulling dotmesh-server docker image...
[...]

This will set up a single-instance cluster on your local machine.

Verify that the dm client can talk to the dotmesh-server:

dm version

If the installation fails, please report an issue. You can also experiment in our online learning environment. Thanks!

See the installation docs for more details, including installing dotmesh on Kubernetes.

getting started guide

Try our hosted tutorial!

Alternatively, try the hello Dotmesh on Docker guided tutorial.

what is a datadot?

A datadot allows you to capture your application's state and treat it like a git repo.

A simple example is to start a PostgreSQL container using a datadot called myapp:

docker run -d --volume-driver dm \
    -v myapp:/var/lib/postgresql/data --name postgres postgres:9.6.6

This creates a datadot called myapp, creates the writeable filesystem for the default master branch in that datadot, mounts the writeable filesystem for the master branch into /var/lib/postgresql/data in the postgres container, and starts the postgres container, like this:

myapp dot with master branch and postgres container's /data volume attached

First, switch to it, which, like cd'ing into a git repo, makes it the "current" dot -- the dot which later dm commands will operate on by default:

dm switch myapp

You can then see the dm list output:

dm list
  DOT      BRANCH  SERVER   CONTAINERS  SIZE       COMMITS  DIRTY
* myapp    master  a1b2c3d  /postgres   40.82 MiB  0        40.82 MiB

The current branch is shown in the BRANCH column and the current dot is marked with a * in the dm list output.

what's next?