if there is no Validator associated with a schema Field it generates illegal json #50

yanfali · 2016-11-29T02:52:36Z

e.g. "authentication": schema.Field{
Description: "Ignored authentication token",
Required: false,
},

Is perfectly valid go code, and the validator runs without complaint, but for the JSON schema it generates illegal JSON Schema. Should we make this a run time error if we discover it in the JSONSchema or in schema?

rs · 2016-11-29T05:54:39Z

Why is it invalid?

yanfali · 2016-11-29T06:04:05Z

Extra comma. I think we can fix that part but I'm wondering if we shouldn't flag it as an error? Is a schema field without a validator?

…

On Mon, Nov 28, 2016, 21:54 Olivier Poitrey ***@***.***> wrote: Why is it invalid? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACK5U--KxuKC6X0zdxH71ADOo7SQMMMeks5rC74ggaJpZM4K-gm9> .

rs · 2016-11-29T06:53:39Z

Wouldn't it be useful to let a field be of any type/format?

yanfali · 2016-11-29T09:19:34Z

Not sure that's legal in json schema and also defeats the point of schema as a validator. I'm not sure what the use case you have in mind is.

…

On Mon, Nov 28, 2016, 22:53 Olivier Poitrey ***@***.***> wrote: Wouldn't it be useful to let a field be of any type/format? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACK5U6uGB38vDqJkp3l38H86jWIpFokxks5rC8v0gaJpZM4K-gm9> .

rs · 2016-11-29T09:30:29Z

No particular use-case. We should certainly throw an error, you're right.

smyrman · 2016-11-29T09:56:15Z

It's perfectly legal to have a field of an arbitrary type in JSON Schema - even an empty dictionary is a valid JSON Schema!

smyrman · 2016-11-29T10:09:34Z

I would recommemd dealing with the comma situation, as the way we add commas feels quite hard-coded and as this issue proves, perhaps a bit error prone. Would it be possible to have anorther look at what the JSON package is doing? Or perhaps even utilize the JSON package for tag serialization?

One usecase I see for not specifying a validation for one particular field in a Schema, would be a logging API, or perhaps a job queue with arbitrary input parameters.

smyrman · 2016-11-29T10:34:50Z

Thinking out loud, perhaps one way could be to make it the responsibility of the "writer" (e.g. call it objectEncoder, arrayEncoder) to know when an object start, printe the leading/trailing { or [ and the commas between key/value pairs.

yanfali · 2016-11-29T17:49:13Z

I think fixing the bug first would be good, then if you feel strongly enough to rewrite the serializer then open a separate issue. While it may be legal to generate a typeless schema I'm not really sure what that purpose it would serve other than to add to confusion.

yanfali · 2016-11-29T17:51:39Z

My initial point is that using schema without a validator appears to be an incorrect usage of the API and should probably be considered an error by something like the JSONSchema. If you want the validator to be a wildcard wouldn't something like AnyOf be a better choice?

@rs do you consider a missing validator as an incorrect API usage or a valid use case?

rs · 2016-11-29T17:54:07Z

Yes in the current state of the code, we should not allow validator-less fields. It is easier to reopen that later if we feel like implementing this use-case.

yanfali · 2016-11-29T20:07:13Z

Thanks I'll try and work up a pull request

…

On Tue, Nov 29, 2016, 09:54 Olivier Poitrey ***@***.***> wrote: Yes in the current state of the code, we should not allow validator-less fields. It is easier to reopen that later if we feel like implementing this use-case. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACK5U5lwVQEkHajvLnq0TdpuPlChmra9ks5rDGbBgaJpZM4K-gm9> .

smyrman · 2016-12-01T04:27:29Z

If you are going to raise an error with a nill validatior, please consider to at least make it a NotYetImplemented type error...

In my opinion, the use case seams completely valid. Even more so if nill validatiors are already allowed by the rest of the rest-layer code. @rs, is this the case?

To go more in depth with the use-case, consider a job queue API with:

Objekt task:

priority (int)
function (string)
parameters (any valid JSON)

Solving it with AnyOf would then require quite ugly code...

Cheers

rs · 2016-12-01T04:56:29Z

I'm not sure the current code expect a field with not validator.

yanfali · 2016-12-01T05:16:13Z

Feel free to implement a don't care validator. I think that solves your use case. This is an incorrect usage of the API, not a semantic issue. This could be solved by implementing a new validator that explicitly outlines the meaning rather than relying on a implicit behavior. I was actually thinking about implementing this in the compile phase.

…

On Wed, Nov 30, 2016, 20:56 Olivier Poitrey ***@***.***> wrote: I'm not sure the current code expect a field with not validator. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACK5U_PLiL7graF-UiIp_4qPQrOOuaMxks5rDlN_gaJpZM4K-gm9> .

smyrman · 2016-12-01T08:57:05Z

Idomatic Go programs actually has a good history for making the nil value useful. Anyway, let's discuss this in a separate issue as you suggest.

When a validator hasn't been set the JSONSchema emits invalid JSON. - add a test case replicating the issue - extend errWriter to track properties written - add a Comma helper which emits commas when at least 1 property has been written - extract function from encoding loop serializeField - change validator check to test validator for nil instead of just errors - errWriter takes care of errors

- if both the Validator and the Schema have not been set, consider this an error and report it during validation.

When a validator hasn't been set the JSONSchema emits invalid JSON. - add a test case replicating the issue - extend errWriter to track properties written - add a Comma helper which emits commas when at least 1 property has been written - extract function from encoding loop serializeField - change validator check to test validator for nil instead of just errors - errWriter takes care of errors

When a validator hasn't been set the JSONSchema emits invalid JSON. - Add a test case replicating the issue - Create `fieldWriter` type which embeds `errWriter` as field tracking behavior is orthogaonal to `errWriters` purpose - Extract function from encoding loop `serializeField` - Change validator check to test validator for nil instead of just errors - ErrWriter takes care of errors

smyrman · 2016-12-13T21:26:59Z

@yanfali, is this now fixed?

yanfali · 2016-12-13T21:30:52Z

Should be, there's a test that confirms the issue in the PR.

smyrman · 2016-12-13T21:35:26Z

Let's close it then:-)

smyrman mentioned this issue Dec 1, 2016

[jsonschema] Rethink how jsonschema deals with encoding internally #51

Closed

yanfali pushed a commit to awakesecurity/rest-layer that referenced this issue Dec 10, 2016

Issue rs#50 - detect improperly constructed Fields

570e045

- if both the Validator and the Schema have not been set, consider this an error and report it during validation.

rs closed this as completed Dec 14, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

if there is no Validator associated with a schema Field it generates illegal json #50

if there is no Validator associated with a schema Field it generates illegal json #50

yanfali commented Nov 29, 2016

rs commented Nov 29, 2016

yanfali commented Nov 29, 2016 via email

rs commented Nov 29, 2016

yanfali commented Nov 29, 2016 via email

rs commented Nov 29, 2016

smyrman commented Nov 29, 2016

smyrman commented Nov 29, 2016

smyrman commented Nov 29, 2016

yanfali commented Nov 29, 2016

yanfali commented Nov 29, 2016

rs commented Nov 29, 2016

yanfali commented Nov 29, 2016 via email

smyrman commented Dec 1, 2016 •

edited

Loading

rs commented Dec 1, 2016

yanfali commented Dec 1, 2016 via email

smyrman commented Dec 1, 2016

smyrman commented Dec 13, 2016

yanfali commented Dec 13, 2016

smyrman commented Dec 13, 2016

if there is no Validator associated with a schema Field it generates illegal json #50

if there is no Validator associated with a schema Field it generates illegal json #50

Comments

yanfali commented Nov 29, 2016

rs commented Nov 29, 2016

yanfali commented Nov 29, 2016 via email

rs commented Nov 29, 2016

yanfali commented Nov 29, 2016 via email

rs commented Nov 29, 2016

smyrman commented Nov 29, 2016

smyrman commented Nov 29, 2016

smyrman commented Nov 29, 2016

yanfali commented Nov 29, 2016

yanfali commented Nov 29, 2016

rs commented Nov 29, 2016

yanfali commented Nov 29, 2016 via email

smyrman commented Dec 1, 2016 • edited Loading

rs commented Dec 1, 2016

yanfali commented Dec 1, 2016 via email

smyrman commented Dec 1, 2016

smyrman commented Dec 13, 2016

yanfali commented Dec 13, 2016

smyrman commented Dec 13, 2016

smyrman commented Dec 1, 2016 •

edited

Loading