Implement more shareable serialization logic for config.json

[nathanleclaire]

This is a proposal to deal with a variety of problems around sharing configurations from computer to computer.

Problem

Currently if one creates a server using Docker Machine, a new folder is created on their computer containing config.json (description of the machine) and secrets associated with that machine. To use this same configuration on another computer is not straightforward, largely because moving the folder to the same path on another machine does not guarantee that the OS paths will be translated correctly.

For instance, a user might have their CaCertPath at /Users/nathanleclaire/.docker/machine/certs/ca.pem, but on another computer the StorePath might be C:\Users\nathan\.docker.

Proposal

This could potentially be mitigated by implementing a custom type with custom JSON marshal and unmarshal logic using Golang's JSONMarshaler and JSONUnmarshaler interfaces.

This would allow us to have a "universal translation layer" for pathing. Likewise,

Code speaks louder than words, so here's an example:

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "strings"
)

type FSPath string

var machineDir = "/Users/nathanleclaire/.docker/machine"

func (fspath FSPath) MarshalJSON() ([]byte, error) {
    stringForm := string(fspath)
    stringForm = strings.Replace(stringForm, machineDir, "{{.MachineDir}}", -1)
    return []byte(fmt.Sprintf(`"%s"`, stringForm)), nil
}

func (fspath *FSPath) UnmarshalJSON(data []byte) error {
    stringForm := string(data)
    *fspath = FSPath(strings.Replace(stringForm, "{{.MachineDir}}", machineDir, -1))
    return nil

}

func main() {
    var caCertPath FSPath
    caCertPath = "/Users/nathanleclaire/.docker/machine/certs/ca-cert.pem"
    caCertJSON, err := json.Marshal(caCertPath)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Persisted form: ", string(caCertJSON))

    var loadedCaCertPath FSPath
    if err := json.Unmarshal(caCertJSON, &loadedCaCertPath); err != nil {
        log.Fatal(err)
    }

    fmt.Println("Loaded form: ", loadedCaCertPath)
}

Output:

Persisted form:  "{{.MachineDir}}/certs/ca-cert.pem"
Loaded form:  "/Users/nathanleclaire/.docker/machine/certs/ca-cert.pem"

Link to playground version: http://play.golang.org/p/fbR-k8nItm

For instance, we could use this to normalize the paths to always have the same type of slashes like mentioned in #2509

If this is implemented properly, we can ensure that the user's StorePath will always be set correctly when loaded, no matter where the store they happened to move their machine folder to is.

Potential issues

Seems like this isn't a particularly risky move, but the most annoying bit would be to update all of our code to use FSPath instead of vanilla string, which is certainly a decent chunk of overhead and decently risky given some issues around serialization and migration that we've had in the past.

Future

One of the reasons I think this could be nice to implement is that it could set a precedent for doing this in other areas as well. For instance, to keep API credentials encrypted when they are on disk and prompt for a password before decrypting them in the Unmarshal step (as just one suggestion of many possibilities).

Related issues / PRs:

• enhancement: normalize file paths in config.json to use forward slashes #2509
• Attach to existing machine from another client #1328

cc @docker/machine-maintainers What do you think about this solution?

[Kosta-Github]

👍

[askb]

👍 definitely good to have! @nathanleclaire Just wondering if this there would be a separate / sub-cmd on the lines of dm export-config for exporting the machine configuration or is this generally applicable during machine creation and also handle migrating config for existing machines ?

[nathanleclaire]

Just wondering if this there would be a separate / sub-cmd on the lines of dm export-config for exporting the machine configuration or is this generally applicable during machine creation and also handle migrating config for existing machines ?

It will be an "internal feature" at first, but is a first step to having an artifact that one can move from computer-to-computer and it keeps working.

[fsoppelsa]

+1 also to dm export-config

[kunalkushwaha]

+1

[srinath-imaginea]

+1 just ran into this. Would be great to have an export/import config command.

[Marcpepe]

+1

[ghost]

Any updates on this?

[sukrit007]

👍

[brandontamm]

+1 Bump on this - anyone have an update on this?

[monodeep12]

Any update on this feature?

[mahnunchik]

Any news?

[mahmoud-samy]

any news?

[brandontamm]

not the cleanest or most secure, but RSYNC the whole .docker folder on your machine and it's a replica HOWEVER there's not really a good way to do this for mobile devices that I've found...

[brandontamm]

https://github.com/lra/mackup

[mahmoud-samy]

👍