Assets 2

Significant changes that went into this release:

  • dotmesh#461 - Object storage integration (set of object versions) - Implemented S3 integration with dotmesh

  • dotmesh#463 - Support CVv2 mode in dotmesh with sentinels to stabilise pod lifecycles

  • dotmesh#337 - Faster dm list using a new API call, DotmeshRPC.ListWithContainers

  • dotmesh#465 - Make the operator log the logs of failed server pods

  • dotmesh#466 - After restarting dotmesh-server-inner, all dots go into "failed, never coming back"

  • dotmesh#463 - Remove ownerref on pods so they can be controlled by the dotmesh operator

  • dotmesh#474 - Remove collaborator rpc endpoint

  • dotmesh#475 - Add UserFromName RPC method

  • dotmesh#476 - Add UpdateUserPassword RPC method

  • dotmesh#481 - Add UpdateUserEmail RPC endpoint

  • dotmesh#482 - Add AuthenticatedUser RPC endpoint

  • dotmesh#420 - Fix, after a dm cluster reset, volumes lose their zfs-ness

  • dotmesh#478 - Add a global --verbose option that displays the JSONRPC requests and responses

  • dotmesh#479 - Allow JSON string as params passed into dm debug

  • dotmesh#490 - Newly handed out docker volume mounts can refer to deleted dots

  • dotmesh#489 - List-like RPCs should refer to the registry rather than local fsMachine state

  • dotmesh#495 - Dot name restrictions mean we can't clean up randomly generated docker volumes

  • dotmesh#497 - Refactor, split up statemachines, types, etc. into multiple files

  • dotmesh#460 - Don't show replication status errors if non-admin user.

  • dotmesh#493 - Recast retry loop to keep going until it finds definite evidence of success or failure.

  • dotmesh#467 - Bazel for build

  • dotmesh#496 - Use Gitlab registry

  • dotmesh#440 - Support ZFS 0.7 (in particular, this fixes dotmesh on Ubuntu 18.04)

  • dotmesh#528 - Fix: pool names can be stupidly long

  • dotmesh#532 - ZFS detection logic makes fragile assumptions

  • dotmesh#534 - Fix ZFS errors in pushes

  • Tidied up source files, directories did various bits of refactoring.

  • Numerous stability fixes for acceptance tests for dotmesh on kubernetes using dind clusters; added cleanup strategies for test runs for CI and local runs, made tests crash faster, fixed networking flakes in tests and many more.

  • Stability fixes for production usage with kubernetes, with explicit mount points for dots stored in ZFS.

Install instructions:

Assets 2
  • dotmesh#428 - Support specifying custom dotmesh ports to connect to in dotmesh-server and client.
  • dotmesh#449 - Fixed Production failure: Dotmesh server pods ending up in Completed state
  • dotmesh#450 - Handle unrecoverable errors with a 'failedState'
  • dotmesh#457 - Fixed dm client adds in the namespace twice on requests to the remote
  • dotmesh#401 - Implemented support for PVPerNodeMode on dotmesh (feature currently in beta, being tested)
  • dotmesh#462 - Update progress bar with total size when we get it from the TransferPoll request
  • dotmesh#422 - Fixed various goroutine leaks in dotmesh
  • dotmesh#455 run a samba container inside vagrant for easy Mac based code editing
  • FIX: At least we can stop reporting stale data in 'dm dot show' and the web UI and such.

Install instructions:

Jun 12, 2018

@alaric-dotmesh alaric-dotmesh released this May 29, 2018 · 181 commits to release-0.5 since this release

Assets 2
  • dotmesh#305 - Prometheus instrumentation of the Dotmesh server.
  • dotmesh#401 - Framework for an upcoming feature, PV-per-node mode in Kubernetes, including a lot of infrastructure to test it in CI!
  • dotmesh#418 - Improvements to our CI infrastructure, and a fix to a deletion edge case we found thanks to those improvements.
  • dotscience#3 (API BREAKING CHANGE): The Commit API call now accepts arbitrary key=value metadata in the request, and returns the commit ID.

Install instructions:

@lukemarsden lukemarsden released this May 16, 2018 · 232 commits to release-0.5 since this release

Assets 2
  • Fix #352 - make divergence recovery more reliable
  • Make subdot roots writeable by all users to support container images which run as non-root users
  • Implement #305 - initial Prometheus instrumentation of dotmesh-server
  • Fix #404 - make push/clone/pull more reliable
  • Stop spewing binary data into the logs
  • Fix #414 - show correct status for branches in dm dot show
  • Fix #403 and #416 - stop pushed dots getting stuck in pushPeerState
  • Fix #182 - use go1.10 build caching in containerized builds to bring build times down by 3x
  • WIP on #401 to implement PV-per-node mode (not tested, documented or enabled by default yet)
  • Fix #373 - always show the progress bar even for very fast transfers

Install (docker, kubernetes):

@prisamuel prisamuel released this May 3, 2018 · 297 commits to release-0.5 since this release

Assets 2

Implemented #344 - Write a basic dotmesh operator that implement our current Daemonset behaviour.

Various stability fixes to state machines, error handling and debug utilities:
#384 - debug dump RPC
#352 - re-convergence after divergence (still a WIP, probably not release-notes-ready yet)
QUICK FIX didn't get an issue - handle "mount" requests in activeState for quicker startup of new dots
#389: Fixed missing error handling in handoffState, curing a possible deadlock

Note specific upgrade instructions for upgrading etcd and etcd operator in this release:

Install (docker, kubernetes):

Assets 2

Fix #354 - upgrade etcd operator to 0.8.4.
Fix #372 - dm cluster backup-etcd and dm cluster restore-etcd commands to enable backups & restores.
Fix #353 - dm dot show command now shows replication status.

Note specific upgrade instructions for upgrading etcd and etcd operator in this release:

Install (docker, kubernetes):

Assets 2

Fix #352 - converge diverged branches when replicating
Fix #325 - mask API keys in error messages and logs
Fix #358 - deletion attempts cause deadlock
Fix #86 - support receiving a filesystem that had been deleted

Install (docker, kubernetes):

@lukemarsden lukemarsden released this Mar 25, 2018 · 21 commits to release-0.3 since this release

Assets 2

Fix #339 - intermittent errors when pushing to multi node clusters.
Fix #351 - make dotmesh server logs show up in kubernetes
Fix #303 - make dm cluster init idempotent

Install (docker, kubernetes):

@danthebaker danthebaker released this Mar 21, 2018 · 54 commits to release-0.3 since this release

Assets 5

Dotmesh release-0.3.1

Dotmesh is a system for capturing, organizing and sharing application states. The first release of Dotmesh offers support for managing dots on Kubernetes and plain Docker clusters, and interoperability with the Dothub.

This release includes the binary

  • dm : The dotmesh client, a git-like CLI for interacting with datadots.

The client can be used to install dotmesh-server, assuming you have Docker installed and your user account has access to the Docker daemon.

$ dm cluster init

This pulls the dotmesh server for this release.

  • dotmesh-server : The server-side component for storing datadots. This can be run on individual servers or a cluster.

The dotmesh server also package dynamic provisioner, and flex-volume driver. Dotmesh uses etcd, and ZFS under the hood to manage and store stateful information.

More information on the command line usage, features, installation instructions and concepts can be found at the Dotmesh Docs

Current binaries support OSX and Linux (nixOS, Ubuntu 16.04 LTS and CentOS 7.4).

For more information about product support, team, and blog please head over to

For a list of current issues

Changes in this version

  • dotmesh#317: Improved speed of cloning/pulling from remote clusters
  • dotmesh#297: Switch dotmesh port from 6969 to 32607 and switch kubernetes hostPort to nodePort