Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: encoding/json: Add Support of Simple Validation Tags #16426

Closed
stonelgh opened this issue Jul 19, 2016 · 6 comments

Comments

Projects
None yet
7 participants
@stonelgh
Copy link

commented Jul 19, 2016

Abstract

This document proposes to support simple validation tags in the encoding/json module.

Background

It's very common for the 2 ends, which communicate in JSON, to validate the received data before processing it. Current approaches include making the validation by introducing a schema validation package or making an in-line validation. Either solution has its shortcomings. A schema validation package seems too heavy for a JSON message containing only simple validations and is not efficient because it causes an extra decoding cycle of the message. An in-line validation will need extra efforts to maintain when the message structure changes.

Proposal

Simple validations can be specified in the tags of corresponding Go's struct. When json.Unmarshal is called to decode the data, these tags will be considered and the data passed in will be checked against them. If one of the validations fails, json.Unmarshal will return with an error describing this situation.

For instance, given the Go struct:

type Message struct {
    Type    int      `json:"type,required" json.enum:"1,2,3"`
    Content string   `json:"content" json.len:"[5,20]"`
}

only a JSON object with a type field, which is of integer type and has a value between 1 and 3, and an optional content field, which is of string type and has a value containing 5 to 20 characters, can pass the validation.

Suggested tags are:

Tag Value Effect Scope Usage Effect
noExtraFields - struct json="[...],noExtraFields" the encoded data cannot contain extra fields other than listed in the struct
required - field json="[...],required" fields with this tag must appear in the encoded data
json.default field default value field json.default="..." provide a default value for that field if missing in the encoded data
json.enum value list seperated by ',' field json.enum="value1,..." restrict the field value to be one of them
json.pattern a regular expression field json.pattern="..." restrict the field value must match the given regular expression
json.multipleOf a number field json.multipleOf="..." only valid to integer fields; restrict the field value must be a multiple of the given number
json.range a range field json.range="[min,max]" only valid to numeric fields; [ or ] means inclusive, ( or ) means exclusive; restrict the field value must be within the given range
json.len a range field json.range="[min,max]" only valid to string or array fields; [ or ] means inclusive, ( or ) means exclusive; restrict the field length must be within the given range
json.format date-time, email, hostname, ipv4, ipv6 or uri field json.format="..." restrict the field value must be in the given format

Impact

The proposal won't break current behaviours. It only has little performance penalty and the implementation will be a little bit complex. It will also bring in package dependency to the regexp package if the pattern or format tag is supported.

References

JSON Schema Validation

@quentinmit quentinmit added this to the Proposal milestone Jul 19, 2016

@quentinmit

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2016

I don't understand why this needs to be done in encoding/json; surely you could walk the decoded struct and perform the same validation separately with a different package, no?

@minux

This comment has been minimized.

Copy link
Member

commented Jul 20, 2016

I don't think we should put any forms of verification built-in in
encoding/json. It won't target the widest audiences.

My counterproposal:
We introduce a encoding.Validator interface, and if the just unmarshaled
field supports this interface, encoding/* package will call it and return
the error if the verification failed.

However, we need to be careful not to make existing types accidentally
implement the new Validator interface.

This also somewhat related to the recent node-level error message issue
(#16433) on encoding/json. We could pass location information to the
validator.

@minux

This comment has been minimized.

Copy link
Member

commented Jul 21, 2016

@stonelgh

This comment has been minimized.

Copy link
Author

commented Jul 21, 2016

Thanks for your timely response!

surely you could walk the decoded struct and perform the same validation separately with a different package, no?

No, it is not possible in some situations without decoding the data twice. For instance, how do you perform the required/noExtraFields validation by walking the decoded struct?

Any interest in the encoding.Validator interface? I could write a formal proposal if there are enough interest.

Although the encoding.Validator interface you suggested does not mean the same thing, it does provide new thoughts to solve similar problems, and gives more flexcibility. I support you. Please go ahead with a new proposal!

Back to the proposal I suggested, if it cannot be accepted, I will make an implementation in a separate package. I believe that more people will benefit from it.

@pciet

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2016

I'm all for expanded JSON specification reduction utility in these tags that are already not that intuitive. Seems like a decent place to add validations since the encoding/json package already has to inspect and reflect on the entire encoded string's data.

Although a more pure and simple JSON package in the standard library, along with an information and validation companion package, could make sense.

@robpike

This comment has been minimized.

Copy link
Contributor

commented Sep 12, 2016

This is a potentially very large change; the schema document is almost as long as the whole Go spec. It makes more sense to do this in an external JSON package, not in the Go core libraries.

@robpike robpike closed this Sep 12, 2016

@golang golang locked and limited conversation to collaborators Sep 12, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.