Add metadata to manifest & warn on unknown fields #528

darkowlzz · 2017-05-08T15:16:47Z

Use manifest buffer to create a map of TomlTree and find all the
unknown fields in the tree. Raise warnings for unknown fields.
For "metadata" field, ensure it's of map type, else raise warning.

Fixes #506

sdboyer · 2017-05-08T18:30:51Z

manifest.go

+						}
+					}
+					if !isValidMetadata {
+						internal.Logf("WARNING: metadata should consist of key-value pairs")


We're probably going to end up wanting to reuse this func elsewhere, in places where it's not a good idea to log directly. So, instead of directly logging the issues, could we switch to a ([]error, error) return type, and record all the problems we detect as items in the first slice?

- Use manifest buffer to create a map of TomlTree and find all the unknown fields in the tree. Raise warnings for unknown fields. - For "metadata" field, ensure it's of map type, else raise warning.

- Modifies ValidateManifest() to return []error, as a collection of all the errors, instead of logging directly. - Adds tests for ValidateManifest().

darkowlzz · 2017-05-08T21:15:23Z

Changed validateManifest() return type to ([]error, error).
Added tests for validateManifest().

sdboyer

just a bit more...

sdboyer · 2017-05-09T00:54:31Z

manifest_test.go

+		{
+			tomlString: `
+			[[dependencies]]
+        name = "github.com/foo/bar"


can we also validate that there aren't any extra fields within [[dependencies]] or [[overrides]] items?

darkowlzz · 2017-05-09T03:43:51Z

@sdboyer thanks to that extra bit, I found that the checks can be improved by just using type assertion. Once a property is TOML array of tables ([[some-name]]), TOML ensures that the content has to be map (key-value pair) :)

Added checks for invalid fields in [[dependencies]] or [[overrides]].

- Improves property validation by checking for TOML array of tables. - Adds checks for invalid fields in dependencies and overrides.

sdboyer · 2017-05-09T05:00:40Z

@darkowlzz hmm - looking at the TOML spec a bit, and poking at the PR, I'm confused - array of tables isn't actually the same thing as just a plain table, because you can specify them multiple times. And we don't want that to be possible - there should be exactly one allowed [metadata] section per area - whether that's at the top-level, or within each subtable (e.g. a [[dependencies]] subtable.

Related - let's also make sure a the test exercises the possibility of nesting metadata within e.g. a [[dependencies]].

- Makes metadata TOML table. - Allows metadata table in "dependencies" and "overrides".

darkowlzz · 2017-05-09T05:51:52Z

@sdboyer great! now that makes things more clear :)

metadata is now a TOML table [metadata] (map).
metadata can be added within subtables of [[dependencies]] and [[overrides]], only as TOML table (map). Added the tests for the same.

sdboyer · 2017-05-09T17:42:47Z

Awesome, this looks great! Thanks so much 😄 🎉

mdelder · 2017-05-09T17:56:06Z

It appears that this PR may have triggered a regression. Consuming the latest, I see:

# github.com/golang/dep
/go/src/github.com/golang/dep/manifest.go:109: undefined: internal.Logf
# github.com/golang/dep
/go/src/github.com/golang/dep/manifest.go:109: undefined: internal.Logf
/usr/local/bin/kube-tlssec-prep.sh: line 22: dep: not found
/usr/local/bin/kube-tlssec-prep.sh: line 23: dep: not found

From:

go get github.com/golang/dep

sdboyer · 2017-05-09T18:24:23Z

@mdelder it did indeed - thanks for responding quickly here. I'm updating things now.

…known-manifest-fields Add metadata to manifest & warn on unknown fields

googlebot added the cla: yes label May 8, 2017

sdboyer suggested changes May 8, 2017

View reviewed changes

Add metadata to manifest & warn on unknown fields

878ae28

- Use manifest buffer to create a map of TomlTree and find all the unknown fields in the tree. Raise warnings for unknown fields. - For "metadata" field, ensure it's of map type, else raise warning.

darkowlzz force-pushed the 506-add-metadata-warn-unknown-manifest-fields branch 2 times, most recently from 64abf1d to 90656bc Compare May 8, 2017 21:08

Return []error from ValidateManifest and add tests

8129217

- Modifies ValidateManifest() to return []error, as a collection of all the errors, instead of logging directly. - Adds tests for ValidateManifest().

darkowlzz force-pushed the 506-add-metadata-warn-unknown-manifest-fields branch from 90656bc to 8129217 Compare May 8, 2017 21:14

sdboyer suggested changes May 9, 2017

View reviewed changes

darkowlzz force-pushed the 506-add-metadata-warn-unknown-manifest-fields branch 4 times, most recently from 1bd28e6 to 831aab0 Compare May 9, 2017 04:46

Improve manifest validation

e518855

- Improves property validation by checking for TOML array of tables. - Adds checks for invalid fields in dependencies and overrides.

darkowlzz force-pushed the 506-add-metadata-warn-unknown-manifest-fields branch from 831aab0 to e518855 Compare May 9, 2017 05:00

Replace metadata array of tables with just table

c588d9e

- Makes metadata TOML table. - Allows metadata table in "dependencies" and "overrides".

sdboyer approved these changes May 9, 2017

View reviewed changes

sdboyer merged commit b3bd16e into golang:master May 9, 2017

sdboyer mentioned this pull request May 9, 2017

Restore warning logging from #528 #531

Closed

ibrasho pushed a commit to ibrasho-forks/dep that referenced this pull request May 10, 2017

Merge pull request golang#528 from darkowlzz/506-add-metadata-warn-un…

0a2a7c5

…known-manifest-fields Add metadata to manifest & warn on unknown fields

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add metadata to manifest & warn on unknown fields #528

Add metadata to manifest & warn on unknown fields #528

darkowlzz commented May 8, 2017

sdboyer May 8, 2017

darkowlzz commented May 8, 2017

sdboyer left a comment

sdboyer May 9, 2017

darkowlzz commented May 9, 2017

sdboyer commented May 9, 2017

darkowlzz commented May 9, 2017

sdboyer commented May 9, 2017

mdelder commented May 9, 2017

sdboyer commented May 9, 2017

Add metadata to manifest & warn on unknown fields #528

Add metadata to manifest & warn on unknown fields #528

Conversation

darkowlzz commented May 8, 2017

sdboyer May 8, 2017

Choose a reason for hiding this comment

darkowlzz commented May 8, 2017

sdboyer left a comment

Choose a reason for hiding this comment

sdboyer May 9, 2017

Choose a reason for hiding this comment

darkowlzz commented May 9, 2017

sdboyer commented May 9, 2017

darkowlzz commented May 9, 2017

sdboyer commented May 9, 2017

mdelder commented May 9, 2017

sdboyer commented May 9, 2017