This is large and exciting release with a number of new features, including network forwarding and session orchestration! The goal of this release is to enable developers to create remote containerized development environments that work with their local tools. For more information, check out the recently renovated project site at https://mutagen.io.
Please bear in mind that network forwarding and session orchestration are still a little experimental and subject to change, so please report any feedback or issues you encounter on the issue tracker or Spectrum chat.
This release includes the following changes from the v0.9.x series:
- Added support for port forwarding between endpoints, including TCP and Unix domain socket forwarding (see below)
- Added support for project-based session orchestration (see below)
- Switched to YAML for configuration (see below)
- Legacy TOML synchronization configurations are still supported, but deprecated
- Reorganized Mutagen command structure to account for expanded functionality
- Synchronization session management commands are still available as hidden commands directly under the root command in order to avoid breaking existing scripts and tools
- Sessions can now be created in a paused state using the
- Sessions can now have user-friendly names assigned using the
- Session specifications must now be either a session name or session identifier (fuzzy matching is no longer supported)
- Added the ability to customize the Mutagen data directory location using the
MUTAGEN_DATA_DIRECTORYenvironment variable, allowing for isolated daemon instances (or just simple relocation of the directory)
- Modified session management commands to automatically start the daemon if not running
- This behavior can be disabled by setting the environment variable
- This behavior can be disabled by setting the environment variable
- Added logging infrastructure
- Switched the default scan mode to
With v0.10.0, Mutagen adds support for network forwarding sessions. These sessions function very similarly to Mutagen's synchronization sessions, with a few necessary differences.
Forwarding sessions are created using the
mutagen forward create command:
mutagen forward create <source> <destination>
Unlike with synchronization, where endpoints are referred to as "alpha" and "beta", forwarding endpoints are referred to as "source" and "destination" since network forwarding is, by its nature, unidirectional.
Forwarding endpoint URLs are similar to synchronization endpoint URLs, except that they encode a network address instead of a filesystem path.
<network-endpoint> has the form
<address> components allow the same values as the
address arguments (respectively) of Go's
net.Listen functions, except that
<protocol> is restricted to
As with synchronization, each endpoint URL can be either local or remote, and they can be combined in arbitrary ways. Basic network forwarding might look like:
mutagen forward create tcp:localhost:8080 docker://appcontainer:tcp:localhost:8080
whereas reverse forwarding would look like:
mutagen forward create docker://appcontainer:tcp:localhost:8080 tcp:localhost:8080
You can even do remote-to-remote forwarding, e..g.:
mutagen forward create email@example.com:tcp:[::1]:8080 : docker://appcontainer:tcp:localhost:8080
With Unix domain socket forwarding, you can expose remote resources:
mutagen forward create unix:~/Projects/docker.sock firstname.lastname@example.org:/var/run/docker.sock
You can even mix-and-match protocols:
mutagen forward create unix:~/mysocket.sock email@example.com:tcp:localhost:9090
Forwarding sessions can then be managed in a very similar fashion to synchronization sessions with the following commands:
mutagen forward list
mutagen forward monitor
mutagen forward pause
mutagen forward resume
mutagen forward terminate
To help support Mutagen's growing functionality, configuration has been switched to use YAML instead. The legacy TOML-based configuration (such as
~/.mutagen.toml and files passed to
mutagen sync create with the
-c/--configuration-file flag) will still be supported, but support for new configuration options, including network forwarding options, will not be added to these files.
The Mutagen YAML format looks like:
sync: defaults: mode: ... maxEntryCount: ... maxStagingFileSize: ... probeMode: ... scanMode: ... stageMode: ... symlink: mode: ... watch: mode: ... pollingInterval: ... ignore: paths: - ... - ... vcs: ... permissions: defaultFileMode: ... defaultDirectoryMode: ... defaultOwner: ... defaultGroup: ... forward: defaults: socket: overwriteMode: ... owner: ... group: ... permissionMode: ...
All fields optional and the syntax for field values is otherwise unchanged (except where YAML requires certain values to be quoted).
Mutagen now looks for its global configuration file in
~/.mutagen.yml (falling back to
~/.mutagen.toml, if it exists, for synchronization sessions) and the
-c/--configuration-file flags have been updated to support this YAML-based format.
Building on its new YAML-based configuration, Mutagen is adding experimental support for synchronization and forwarding session orchestration via the
mutagen project command. This lets you create project configurations defining the synchronization and forwarding sessions for your project and have Mutagen automatically manage those sessions. This is particularly powerful when combined with a tool like Docker Compose to set up sessions. This functionality is still in its infancy, so any and all feedback is welcome, though either the Spectrum chat or issue tracker.
In order to use this functionality, you need to create a
mutagen.yml file that contains your session configuration(s). The format for this file is based off of the YAML layout defined above. In addition to a
defaults key, you can specify named sessions with pre-defined endpoint URLs and configuration options. An example of this file can be found in the Mutagen repository, along with a corresponding Docker Compose configuration and underlying Dockerfile.
Using the orchestration infrastructure, your workflow might look something like:
# Start Docker Compose services docker-compose up --build --force-recreate --detach # Start Mutagen project sessions mutagen project start
You can then work interactively in containers by starting a shell with a command like:
docker-compose exec <service-name> /bin/sh
Finally, to shut down the orchestration infrastructure, you can do:
# Terminate Mutagen project sessions mutagen project terminate # Terminate Docker Compose services docker-compose down --rmi=all
Once you have that workflow set up, you can even move the entire development infrastructure off of your laptop using the
DOCKER_HOST environment variable when invoking
mutagen project start, and
docker-compose down (Mutagen will lock in
DOCKER_HOST and other Docker environment variables and doesn't need them re-specified for each command).
mutagen project command also provides other subcommands for managing projects, including:
mutagen project list(to list project sessions and their status)
mutagen project flush(to flush project synchronization sessions)
mutagen project pause(to pause project sessions)
mutagen project resume(to resume paused sessions or sessions that require authentication)
mutagen project terminate(to terminate project sessions)