Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Allow for missing values #60

Closed
scott2b opened this Issue · 8 comments

4 participants

@scott2b

It is not clear why colander strictly enforces the presence of non-required values. There should at least be an 'optional' parameter which would enable validation of schemata with non-required nodes without forcing undesired default or sentinel values.

@mcdonc
Owner

Can you provide a bit of sample code that doesn't behave as you'd like/expect? Does your schema have "missing=" values?

@mcdonc
Owner

No response, closing.

@mcdonc mcdonc closed this
@scott2b

Chris, sorry I did not get back to you on this. It was a weird week and my inbox is kind of piled up. The issue was that missing= is really treated as a default=, meaning that if the value is not provided, then a default is provided. My feeling is that it would be really useful to actually allow for missing values which do not have a default. The primary use case I have is in validating schema for NoSQL or similar solutions where you don't want empty values cluttering up the data. To do this, I am defaulting to None, and then just popping the keys with None values before pushing into a MongoDB.

This is inconvenient but it works. I think this really goes in the category of design choice more than bug.

@mcdonc
Owner

Ah, right! I'l reopen this. It's a reasonable feature request (albeit one we might not be able to get around to in any predictable amount of time). I might suggest something like "missing=colander.drop" for child nodes of sequences and mappings. If subnode of a sequence or mapping deserializes to "colander.drop", the container implementation would just omit it from the deserialization value.

@mcdonc mcdonc reopened this
@scott2b

That seems like a good approach.

@kusut

Request extended.

Say you've implemented colander.drop and people are able to set it on their nodes. Would it be convenient to have a schema level setting to make all nodes dropable?

Here be example:

class Foo(Schema):
    bar = String
    baz = String

The idea is to make this schema works normally as it currently does, and provides a switch to make all nodes droppable when missing.

Maybe give deserialize in _SchemaNode another kwarg:

class _SchemaNode(object):
    deserialize(self, cstruct=null, drop=False)

So it will be like this:

>>> f = Foo()
>>> f.deserialize({'bar': 'abc'}, True) # all missings, in this case baz, will be dropped
{'bar': u'abc'}

Note: maybe raising invalid if all nodes are missing? Another kwarg?

This way, we can use the same schema on both creating and updating an object.

Is this a good idea? Thoughts?

@AndreLouisCaron

+1 for the deserialize(..., drop=True) suggestion. To me, this is not a property of the schema, but rather a filter on the result of deserialization.

@mcdonc
Owner

The missing=colander.drop feature was implemented. I'm going to close this issue as a result. If the suggestions above need further consideration, please open new issues with them attached.

@mcdonc mcdonc closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.