Implement more shareable serialization logic for config.json #2516

Open
nathanleclaire opened this Issue Dec 7, 2015 · 16 comments

Comments

Projects
None yet
@nathanleclaire
Contributor

nathanleclaire commented Dec 7, 2015

This is a proposal to deal with a variety of problems around sharing configurations from computer to computer.

Problem

Currently if one creates a server using Docker Machine, a new folder is created on their computer containing config.json (description of the machine) and secrets associated with that machine. To use this same configuration on another computer is not straightforward, largely because moving the folder to the same path on another machine does not guarantee that the OS paths will be translated correctly.

For instance, a user might have their CaCertPath at /Users/nathanleclaire/.docker/machine/certs/ca.pem, but on another computer the StorePath might be C:\Users\nathan\.docker.

Proposal

This could potentially be mitigated by implementing a custom type with custom JSON marshal and unmarshal logic using Golang's JSONMarshaler and JSONUnmarshaler interfaces.

This would allow us to have a "universal translation layer" for pathing. Likewise,

Code speaks louder than words, so here's an example:

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "strings"
)

type FSPath string

var machineDir = "/Users/nathanleclaire/.docker/machine"

func (fspath FSPath) MarshalJSON() ([]byte, error) {
    stringForm := string(fspath)
    stringForm = strings.Replace(stringForm, machineDir, "{{.MachineDir}}", -1)
    return []byte(fmt.Sprintf(`"%s"`, stringForm)), nil
}

func (fspath *FSPath) UnmarshalJSON(data []byte) error {
    stringForm := string(data)
    *fspath = FSPath(strings.Replace(stringForm, "{{.MachineDir}}", machineDir, -1))
    return nil

}

func main() {
    var caCertPath FSPath
    caCertPath = "/Users/nathanleclaire/.docker/machine/certs/ca-cert.pem"
    caCertJSON, err := json.Marshal(caCertPath)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("Persisted form: ", string(caCertJSON))

    var loadedCaCertPath FSPath
    if err := json.Unmarshal(caCertJSON, &loadedCaCertPath); err != nil {
        log.Fatal(err)
    }

    fmt.Println("Loaded form: ", loadedCaCertPath)
}

Output:

Persisted form:  "{{.MachineDir}}/certs/ca-cert.pem"
Loaded form:  "/Users/nathanleclaire/.docker/machine/certs/ca-cert.pem"

Link to playground version: http://play.golang.org/p/fbR-k8nItm

For instance, we could use this to normalize the paths to always have the same type of slashes like mentioned in #2509

If this is implemented properly, we can ensure that the user's StorePath will always be set correctly when loaded, no matter where the store they happened to move their machine folder to is.

Potential issues

Seems like this isn't a particularly risky move, but the most annoying bit would be to update all of our code to use FSPath instead of vanilla string, which is certainly a decent chunk of overhead and decently risky given some issues around serialization and migration that we've had in the past.

Future

One of the reasons I think this could be nice to implement is that it could set a precedent for doing this in other areas as well. For instance, to keep API credentials encrypted when they are on disk and prompt for a password before decrypting them in the Unmarshal step (as just one suggestion of many possibilities).

Related issues / PRs:

cc @docker/machine-maintainers What do you think about this solution?

@Kosta-Github

This comment has been minimized.

Show comment
Hide comment

👍

@askb

This comment has been minimized.

Show comment
Hide comment
@askb

askb Dec 11, 2015

Contributor

👍 definitely good to have! @nathanleclaire Just wondering if this there would be a separate / sub-cmd on the lines of dm export-config for exporting the machine configuration or is this generally applicable during machine creation and also handle migrating config for existing machines ?

Contributor

askb commented Dec 11, 2015

👍 definitely good to have! @nathanleclaire Just wondering if this there would be a separate / sub-cmd on the lines of dm export-config for exporting the machine configuration or is this generally applicable during machine creation and also handle migrating config for existing machines ?

@nathanleclaire

This comment has been minimized.

Show comment
Hide comment
@nathanleclaire

nathanleclaire Dec 11, 2015

Contributor

Just wondering if this there would be a separate / sub-cmd on the lines of dm export-config for exporting the machine configuration or is this generally applicable during machine creation and also handle migrating config for existing machines ?

It will be an "internal feature" at first, but is a first step to having an artifact that one can move from computer-to-computer and it keeps working.

Contributor

nathanleclaire commented Dec 11, 2015

Just wondering if this there would be a separate / sub-cmd on the lines of dm export-config for exporting the machine configuration or is this generally applicable during machine creation and also handle migrating config for existing machines ?

It will be an "internal feature" at first, but is a first step to having an artifact that one can move from computer-to-computer and it keeps working.

@fsoppelsa

This comment has been minimized.

Show comment
Hide comment
@fsoppelsa

fsoppelsa Dec 12, 2015

Contributor

+1 also to dm export-config

Contributor

fsoppelsa commented Dec 12, 2015

+1 also to dm export-config

@kunalkushwaha

This comment has been minimized.

Show comment
Hide comment
@kunalkushwaha

kunalkushwaha Jan 4, 2016

Contributor

+1

Contributor

kunalkushwaha commented Jan 4, 2016

+1

@srinath-imaginea

This comment has been minimized.

Show comment
Hide comment
@srinath-imaginea

srinath-imaginea Jan 21, 2016

+1 just ran into this. Would be great to have an export/import config command.

+1 just ran into this. Would be great to have an export/import config command.

@Marcpepe

This comment has been minimized.

Show comment
Hide comment

+1

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Feb 16, 2016

Any updates on this?

ghost commented Feb 16, 2016

Any updates on this?

@sukrit007

This comment has been minimized.

Show comment
Hide comment

👍

@brandontamm

This comment has been minimized.

Show comment
Hide comment
@brandontamm

brandontamm Sep 15, 2016

+1 Bump on this - anyone have an update on this?

+1 Bump on this - anyone have an update on this?

@monodeep12

This comment has been minimized.

Show comment
Hide comment
@monodeep12

monodeep12 Dec 10, 2016

Any update on this feature?

Any update on this feature?

@mahnunchik

This comment has been minimized.

Show comment
Hide comment

Any news?

@mahmoud-samy

This comment has been minimized.

Show comment
Hide comment

any news?

@brandontamm

This comment has been minimized.

Show comment
Hide comment
@brandontamm

brandontamm Mar 7, 2018

not the cleanest or most secure, but RSYNC the whole .docker folder on your machine and it's a replica HOWEVER there's not really a good way to do this for mobile devices that I've found...

not the cleanest or most secure, but RSYNC the whole .docker folder on your machine and it's a replica HOWEVER there's not really a good way to do this for mobile devices that I've found...

@brandontamm

This comment has been minimized.

Show comment
Hide comment
@mahmoud-samy

This comment has been minimized.

Show comment
Hide comment

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment