Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use yaml.UnmarshalStrict everywhere #3794

Merged
merged 2 commits into from
Jul 12, 2019
Merged

Conversation

alvaroaleman
Copy link
Contributor

@alvaroaleman alvaroaleman commented Jul 11, 2019

What this PR does / why we need it:

This PR makes us use yaml.UnmarshalStrict everywhere. This has the advantage that unknown fields cause an error instead of being silently dropped.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:

Documentation:

Does this PR introduce a user-facing change?:

[ACTION REQUIRED] Kubermatic now doesn't accept unknown keys in its config files anymore and will crash if an unknown key is present
[ACTION REQUIRED] BYO datacenters now need to be specific in the `datacenters.yaml` with a value of `{}`, e.G `bringyourown: {}`

/assign @zreigz
/assign @nikhita

@kubermatic-bot kubermatic-bot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 11, 2019
@nikhita
Copy link
Contributor

nikhita commented Jul 11, 2019

@alvaroaleman could this lead to compatibility issues with older versions if they persisted unknown fields?

@alvaroaleman
Copy link
Contributor Author

I think we never removed fields so far, we only added stuff. Can you config @mrIncompetent ?

@mrIncompetent
Copy link
Contributor

@alvaroaleman

Can you config @mrIncompetent ?

How do you wan't me to be configured?

@alvaroaleman
Copy link
Contributor Author

/pony typo

@kubermatic-bot
Copy link
Contributor

@alvaroaleman: pony image

In response to this:

/pony typo

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@mrIncompetent
Copy link
Contributor

We replaced AMI with ImageList within the AWS spec.
I assume there might be old files out there which might still contain the AMI field.

So we might want to think about something to ease the operators(The human, not the controller) life in case such an error gets triggered.
Maybe print an error linking to a docs page in case the unmarshal fails?

@alvaroaleman
Copy link
Contributor Author

Maybe print an error linking to a docs page in case the unmarshal fails?

Not sure, I mean the error will contain a "field bla was not found", IMHO that is pretty self-explanatory.

IMHO compatibility is mainly an issue if we would want to allow ppl to use the same config files for multiple versions of Kubermatic, but that is not the case. The config files are supposed to be compatible with exactly one version of Kubermatic. And IMHO the burden of forcing everyone to clean up is not a bad thing at all, since it keeps everything tidy

Goes to the secrets repo and removes unknown field "provider"

@zreigz
Copy link
Contributor

zreigz commented Jul 11, 2019

the API can not start:

{"level":"fatal","time":"2019-07-11T15:01:29.330+0200","caller":"kubermatic-api/main.go:78","msg":"failed to create and initialize providers","error":"failed to load datacenter yaml \"/home/lukasz/go/src/github.com/kubermatic/secrets/seed-clusters/dev.kubermatic.io/datacenters.yaml\": error unmarshaling JSON: while decoding JSON: json: unknown field \"provider\""}

@alvaroaleman
Copy link
Contributor Author

Yes, I am fixing up our datacenters.yaml. Note even our unittests pass with this 😬

@mrIncompetent
Copy link
Contributor

A nice customer experience would be if the customer knows ahead of time if the datacenters.yaml must be checked.
So far we never had a release note stating that the datacenters.yaml must be modified or kubermatic throws errors. Now we require changes on that file and those should be documented in upgrade docs + a release note.

@alvaroaleman
Copy link
Contributor Author

A nice customer experience would be if the customer knows ahead of time if the datacenters.yaml must be checked.

Yes, I absolutely agree. Will add an [Action Needed] release note

@kubermatic-bot kubermatic-bot added release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jul 11, 2019
@zreigz
Copy link
Contributor

zreigz commented Jul 11, 2019

there is another error:

{"level":"fatal","time":"2019-07-11T16:11:26.242+0200","caller":"kubermatic-api/main.go:78","msg":"failed to create and initialize providers","error":"failed to load datacenter yaml \"/home/lukasz/go/src/github.com/kubermatic/secrets/seed-clusters/dev.kubermatic.io/datacenters.yaml\": error unmarshaling JSON: while decoding JSON: json: unknown field \"region\""}

@alvaroaleman
Copy link
Contributor Author

/retest

@alvaroaleman
Copy link
Contributor Author

Tests passed, ptal @mrIncompetent @zreigz

@zreigz
Copy link
Contributor

zreigz commented Jul 12, 2019

/approve no-issue

@kubermatic-bot kubermatic-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 12, 2019
@mrIncompetent
Copy link
Contributor

/lgtm
/approve

@kubermatic-bot kubermatic-bot added the lgtm Indicates that a PR is ready to be merged. label Jul 12, 2019
@kubermatic-bot
Copy link
Contributor

LGTM label has been added.

Git tree hash: c04c48f616dccf4ee2270671861d0d0322b57e29

@kubermatic-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mrIncompetent, zreigz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [mrIncompetent,zreigz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubermatic-bot kubermatic-bot merged commit 64e2dbb into master Jul 12, 2019
@kubermatic-bot kubermatic-bot deleted the yaml-unmarshal-strict branch July 12, 2019 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants