Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract StackConverter from the StackClient #1152

Merged
merged 1 commit into from Jun 29, 2018

Conversation

vdemeester
Copy link
Collaborator

It makes it easier to get the correct stack from a compose config
struct without requiring the client (and thus talking to k8s API)

Signed-off-by: Vincent Demeester vincent@sbr.pm

It makes it easier to get the correct stack from a compose config
struct without requiring the client (and thus talking to k8s API)

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Copy link
Contributor

@silvin-lubecki silvin-lubecki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

_, err := NewStackConverter("v1alpha1")
assert.Check(t, is.ErrorContains(err, "stack version v1alpha1 unsupported"))

_, err = NewStackConverter("v1beta1")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: check the StackConverter returned is not nil?

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
corev1 "k8s.io/client-go/kubernetes/typed/core/v1"
"k8s.io/client-go/rest"
)

// StackClient talks to a kubernetes compose component.
type StackClient interface {
CreateOrUpdate(s stack) error
StackConverter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can really split the 2 interfaces and the 2 implementations? And manipulate 2 objects, one StackClient on one side, and one StackConverter on the other side? I know it will be a more complicated refactoring, so maybe in a follow-up?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point :) I did it that way to not do too much changes but I do agree it would be even better.. Doable in this PR too 😉


func v1beta1FromCompose(stderr io.Writer, name string, cfg *composetypes.Config) (Stack, error) {
cfg.Version = v1beta1.MaxComposeVersion
st, err := v1beta2FromCompose(stderr, name, cfg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is confusing; what's beta2 doing in a beta1 conversion?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just moved code, but mainly, Stack struct holds both the composefile (v1beta1 and the v1beta2 struct) 😅

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @silvin-lubecki because I kinda agree we shouldn't need to do that 👼

return Stack{}, errors.Wrapf(err, "the compose yaml file is invalid with v%s", v1beta1.MaxComposeVersion)
}

st.ComposeFile = string(res)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering if there's consequences of not passing the original file content; is it to normalize the YAML? (if so, we may need some comments explaining)

@vdemeester vdemeester force-pushed the extract-converter branch 2 times, most recently from b28ebcc to f2e6ee6 Compare June 28, 2018 12:59
@silvin-lubecki
Copy link
Contributor

silvin-lubecki commented Jun 28, 2018

Well, LGTM as it is 😉 (+ the tests to improve)

@thaJeztah
Copy link
Member

(commenting here, to prevent the comment from being collapsed if code is updated). Discussing #1152 (comment) with @vdemeester;

What it comes down to is that we try to replicate the swarm (client-side) handling of compose files as much as possible; this means that the actual version (version: 3.x) of the compose file is "ignored", and we try to treat the file as "max version supported by server-side component";

// MaxComposeVersion is the most recent version of compose file Schema supported in v1beta1
const MaxComposeVersion = "3.5"

For example, if a compose-file is used that has version: 3.7, but only uses features that were already available in version 3.5 of the compose file schema, it will be sent as a 3.5 compose-file if no errors are encountered).

The flow is as follows;

  1. Read/parse the file, using the maximum schema that's supported by the CLI (3.7 currently), and convert it into the internal "compose" struct/representation.
    • The file is parsed using the latest version so that syntax features in the latest compose version can be supported
  2. If step 1. was successful; re-encode the internal representation back to a YAML file
  3. Re-parse the YAML file as a 3.5 version, and if that succeeds, we can know it's usable as a 3.5 compose file, and no 3.5+ features were used.

Do we need to put some information about that in the description of FromCompose()? Or should "transcoding" be a separate utility (taking "target version" as argument)?

Note that we were also discussing a limitation in the swarm handling of docker-compose files; unlike command-line options, the compose-file does not label features as being supported by a specific Docker API version, so even if the conversion to the internal format succeeded, actually deploying may fail if the API version doesn't support a feature. Not really sure what the best approach for that would be; annotate options in the compose-file (likely doesn't make sense); perhaps validate the internal representation, and make a swarm-equivalent of warnUnsupportedFeatures() that is "API version aware". Both options may be too much.

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; let's discuss adding more GoDoc in a follow up

@thaJeztah thaJeztah merged commit 7c7c299 into docker:master Jun 29, 2018
@GordonTheTurtle GordonTheTurtle added this to the 18.07.0 milestone Jun 29, 2018
@vdemeester vdemeester deleted the extract-converter branch June 29, 2018 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants