Skip to content
This repository

Allow for missing values #60

scott2b opened this Issue · 8 comments

4 participants

Scott Bradley Chris McDonough kusut André Caron
Scott Bradley

It is not clear why colander strictly enforces the presence of non-required values. There should at least be an 'optional' parameter which would enable validation of schemata with non-required nodes without forcing undesired default or sentinel values.

Chris McDonough

Can you provide a bit of sample code that doesn't behave as you'd like/expect? Does your schema have "missing=" values?

Chris McDonough

No response, closing.

Scott Bradley

Chris, sorry I did not get back to you on this. It was a weird week and my inbox is kind of piled up. The issue was that missing= is really treated as a default=, meaning that if the value is not provided, then a default is provided. My feeling is that it would be really useful to actually allow for missing values which do not have a default. The primary use case I have is in validating schema for NoSQL or similar solutions where you don't want empty values cluttering up the data. To do this, I am defaulting to None, and then just popping the keys with None values before pushing into a MongoDB.

This is inconvenient but it works. I think this really goes in the category of design choice more than bug.

Chris McDonough

Ah, right! I'l reopen this. It's a reasonable feature request (albeit one we might not be able to get around to in any predictable amount of time). I might suggest something like "missing=colander.drop" for child nodes of sequences and mappings. If subnode of a sequence or mapping deserializes to "colander.drop", the container implementation would just omit it from the deserialization value.

Chris McDonough mcdonc reopened this
Scott Bradley

That seems like a good approach.


Request extended.

Say you've implemented colander.drop and people are able to set it on their nodes. Would it be convenient to have a schema level setting to make all nodes dropable?

Here be example:

class Foo(Schema):
    bar = String
    baz = String

The idea is to make this schema works normally as it currently does, and provides a switch to make all nodes droppable when missing.

Maybe give deserialize in _SchemaNode another kwarg:

class _SchemaNode(object):
    deserialize(self, cstruct=null, drop=False)

So it will be like this:

>>> f = Foo()
>>> f.deserialize({'bar': 'abc'}, True) # all missings, in this case baz, will be dropped
{'bar': u'abc'}

Note: maybe raising invalid if all nodes are missing? Another kwarg?

This way, we can use the same schema on both creating and updating an object.

Is this a good idea? Thoughts?

André Caron

+1 for the deserialize(..., drop=True) suggestion. To me, this is not a property of the schema, but rather a filter on the result of deserialization.

Andrew Brookins abrookins referenced this issue in mozilla-services/cornice

Support colander.drop #154

Chris McDonough mcdonc closed this
Chris McDonough

The missing=colander.drop feature was implemented. I'm going to close this issue as a result. If the suggestions above need further consideration, please open new issues with them attached.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.