Fail when extra fields are in the payload

[dorner]

Currently, when passing a datum into the schema validator, the datum will pass even if the datum has more fields than the schema does. This can be dangerous because the client code is assuming that the schema has these values, but they are actually being silently swallowed when they are encoded.

This PR updates the validator so it will fail if any extra fields are passed into it.

[tjwp]

What is the behavior of schema validation in the other language implementations? Does it complain if additional data is present? I'm wary of making the behavior of the ruby implementation different to other languages.

It probably makes sense to open/discuss this on the official avro repo first and then add it here after that discussion.

[dorner]

I know that the node.js one has a toggle to allow or disallow extra fields: https://github.com/mtth/avsc/wiki/API#typeisvalidval-opts. Looks like the default is to allow them.
I can implement something like this as an option if it works. Might be a bit more work to also add that to avro_turf though. I'm concerned about how long it'll take for everything to percolate through. :(

[arron-green]

i verified with python's fastavro that adding an extra field that's not in the schema is also ignored

[tjwp]

I would prefer to have an option to disallow the extra fields, and allow them by default. For this gem, I don't think we want to change default behavior in ways that may not make it into the official gem. But I'm okay with additional, optional functionality

[arron-green]

+1 for optional functionality

[dorner]

All righty. I'll look into changing it this way.

[dorner]

@tjwp updated the code, can you take another look?

[tjwp]

That's the only comment I have. What do you think @jkapell?

[tjwp]

Can this be done more efficiently as something like the following:

(datum.keys.map(&:to_s) - expected_schema.fields.map(&:name)).each do |extra_field|
  result.add_error(path, "extra field '#{extra_field}' - not in schema")
end

[dorner]

@tjwp made the change as requested.

[dorner]

Nice! So for next steps, would you rather I make the PR against the avro gem before merging this in?

[tjwp]

I can run with getting this merged into this gem at this point. Feel free to open an official avro PR in parallel.