Facilitate type conversions

[martijnvermaat]

This might be totally out of your intended scope of Cerberus, and the implementation I have in mind is not 100% pretty, but let me explain my use case :)

Data I want to validate can come in serialized as JSON, but also as HTTP form data, or even query string parameters. In the latter cases, everything is basically a string. You could also consider datetime values in JSON.

I'd like to cast/convert some values (e.g. integers serialized as strings in HTTP form data) in the Validator, such that only one pass over the schema and data is needed and validation rules on the converted value can be used as usual. This conversion should then be done before the other validation rules are applied.

Currently I use the following hack (example) in a Validator subclass to do any conversions in the type rules:

def _validate_type_integer(self, field, value):
    if isinstance(value, basestring):
        try:
            self.document[field] = int(value)
        except ValueError:
            pass
    super(ApiValidator, self)._validate_type_integer(field, self.document[field])

To make sure type rules are applied first, I applied this patch (subclassing or monkey-patching would basically mean duplicating the entire _validate method): martijnvermaat/cerberus@dd0de1f85

List and dictionary types is where it gets more ugly. Some hacking is necessary to make sure converted values end up in the top-level validator document.

After validation, I can then take the data from the validator document field and directly work with the converted values.

The alternative is do the conversion before validation, but that would mean duplicating Cerberus' schema parsing and document traversal.

So this is what I do now, and it works fine, but I'm not happy about having to use the patched _validate method. (In practice I have more intricate data conversions that just strings to integers.)

Other issues I see:

• The naming "validate" no longer accurately describe what's being done.
• You might not like the idea of the input document being modified (solution could be to generate a copy with converted values).

What are your thoughts? Do you see this as a valid use case? I could see this implemented in a cleaner way by having optional _convert_integer etc rules, which are always applied first.

[nicolaiarocci]

Since you are doing two very different things (conversion, and then validation), I would definitely apply the 'separation of concerns' principle here. If the data coming in (the 'string') is supposed to be a dict, you could attempt a conversion with the ast.literal_eval builtin (Py2.6+).

literal_eval(node_or_string)
    Safely evaluate an expression node or a string containing a Python
    expression.  The string or node provided may only consist of the following
    Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
    and None.

I suspect it wouldn't be enough, thought, but it could be a good starting point. In Eve I am using simplejson/json loads method to obtain a dict out of query strings and request form data.

[nicolaiarocci]

Also, since you mention dates in JSON streams, did you check JSON-Datetime? It's a little toy of mine, built when I was experimenting with JSON and conversion issues. The Eve implementation is slightly different though.

[martijnvermaat]

I think you are right and I should separate concerns. Thanks for the pointer to JSON-Datetime, I'll check it out.