Colander is useful as a system for validating and deserializing data obtained via XML, JSON, an HTML form post or any other equally simple data serialization. Colander can be used to:
- Define a data schema
- Deserialize a data structure composed of strings, mappings, and lists into an arbitrary Python structure after validating the data structure against a data schema.
- Serialize an arbitrary Python structure to a data structure composed of strings, mappings, and lists.
Out of the box, Colander can serialize and deserialize various types of objects, including:
- A mapping object (e.g. dictionary)
- A variable-length sequence of objects (each object is of the same type).
- A fixed-length tuple of objects (each object is of a different type).
- A string or Unicode object.
- An integer.
- A float.
- A boolean.
- An importable Python object (to a dotted Python object path).
Colander allows additional data structures to be serialized and deserialized by allowing a developer to define new "types".
Defining A Colander Schema
Imagine you want to deserialize and validate a serialization of data you've obtained by reading a YAML document. An example of such a data serialization might look something like this:
Let's further imagine you'd like to make sure, on demand, that a particular serialization of this type read from this YAML document or another YAML document is "valid".
Notice that all the innermost values in the serialization are strings, even though some of them (such as age and the position of each friend) are more naturally integer-like. Let's define a schema which will attempt to convert a serialization to a data structure that has different types.
For ease of reading, we've actually defined five schemas above, but
we coalesce them all into a single
Person schema. As the result
of our definitions, a
name, which must be a string.
age, which must be deserializable to an integer; after deserialization happens, a validator ensures that the integer is between 0 and 200 inclusive.
- A sequence of
friendstructures. Each friend structure is a two-element tuple. The first element represents an integer rank; it must be between 0 and 9999 inclusive. The second element represents a string name.
- A sequence of
phonestructures. Each phone structure is a mapping. Each phone mapping has two keys:
locationmust be one of
home. The number must be a string.
Schema Node Objects
A schema is composed of one or more schema node objects, each typically of the class :class:`colander.SchemaNode`, usually in a nested arrangement. Each schema node object has a required type, an optional deserialization validator, an optional default, an optional title, an optional description, and a slightly less optional name.
The validator of a schema node is called after deserialization; it
makes sure the deserialized value matches a constraint. An example of
such a validator is provided in the schema above:
validator=colander.Range(0, 200). A validator is not called after
serialization, only after deserialization.
The default of a schema node indicates its default value if a value for the schema node is not found in the input data during serialization and deserialization. It should be the deserialized representation. If a schema node does not have a default, it is considered required.
The name of a schema node appears in error reports.
The title of a schema node is metadata about a schema node that can be used by higher-level systems. By default, it is a capitalization of the name.
The description of a schema node is metadata about a schema node that can be used by higher-level systems. By default, it is empty.
The name of a schema node that is introduced as a class-level attribute of a :class:`colander.MappingSchema`, :class:`colander.TupleSchema` or a :class:`colander.SequenceSchema` is its class attribute name. For example:
The name of the schema node defined via
colander.SchemaNode(..) within the schema above is
The title of the same schema node is
In the examples above, if you've been paying attention, you'll have noticed that we're defining classes which subclass from :class:`colander.MappingSchema`, :class:`colander.TupleSchema` and :class:`colander.SequenceSchema`.
It's turtles all the way down: the result of creating an instance of any of :class:`colander.MappingSchema`, :class:`colander.TupleSchema` or :class:`colander.SequenceSchema` object is also a :class:`colander.SchemaNode` object.
Deserializing A Data Structure Using a Schema
Earlier we defined a schema:
Let's now use this schema to try to deserialize some concrete data structures.
Deserializing A Valid Serialization
schema.deserialize(data) is called, because all the data in
the schema is valid, and the structure represented by
conforms to the schema,
deserialized will be the following:
Note that all the friend rankings have been converted to integers, likewise for the age.
Deserializing An Invalid Serialization
data structure has some problems. The
age is a
negative number. The rank for
t which is not a valid
location of the first phone is
bar, which is not
a valid location (it is not one of "work" or "home"). What happens
when a data structure cannot be deserialized due to a data type error
or a validation error?
deserialize method will raise an exception, and the
clause above will be invoked, causing an error messaage to be printed.
It will print something like:
The above error is telling us that:
- The top-level age variable failed validation.
- Bob's rank (the Friend tuple name
bob's zeroth element) is not a valid number.
- The zeroth phone number has a bad location: it should be one of "home" or "work".
We can optionally catch the exception raised and obtain the raw error dictionary:
This will print something like:
Defining A Schema Imperatively
The above schema we defined was defined declaratively via a set of
class statements. It's often useful to create schemas more
dynamically. For this reason, Colander offers an "imperative" mode of
schema configuration. Here's our previous declarative schema:
We can imperatively construct a completely equivalent schema like so:
Defining a schema imperatively is a lot uglier than defining a schema declaratively, but it's often more useful when you need to define a schema dynamically. Perhaps in the body of a function or method you may need to disinclude a particular schema field based on a business condition; when you define a schema imperatively, you have more opportunity to control the schema composition.
Serializing and deserializing using a schema created imperatively is done exactly the same way as you would serialize or deserialize using a schema created declaratively:
Defining a New Type
A new type is a class with two methods::
serialize converts a Python data structure to a
deserialize converts a value to a Python data
Here's a type which implements boolean serialization and
deserialization. It serializes a boolean to the string
false; it deserializes a string (presumably
but allows some wiggle room for
1) to a boolean value.
Here's how you would use the resulting class as part of a schema:
The above schema has a member named
interested which will now be
serialized and deserialized as a boolean, according to the logic
defined in the
Boolean type class.
Note that the only real constraint of a type class is that its
serialize method must be able to make sense of a value generated
deserialize method and vice versa.
The serialize and deserialize methods of a type accept two values:
node will be the schema node associated
with this type. It is used when the type must raise a
:exc:`colander.Invalid` error, which expects a schema node as its
first constructor argument.
value will be the value that needs to
be serialized or deserialized.
For a more formal definition of a the interface of a type, see :class:`colander.interfaces.Type`.
Defining a New Validator
A validator is a callable which accepts two positional arguments:
value. It returns
None if the value is valid.
It raises a :class:`colander.Invalid` exception if the value is not
valid. Here's a validator that checks if the value is a valid credit
Here's how the resulting
luhnok validator might be used in a
Note that the validator doesn't need to check if the
value is a
string: this has already been done as the result of the type of the
cc_number schema node being :class:`colander.String`. Validators
are always passed the deserialized value when they are invoked.
node value passed to the validator is a schema node object; it
must in turn be passed to the :exc:`colander.Invalid` exception
constructor if one needs to be raised.
For a more formal definition of a the interface of a validator, see :class:`colander.interfaces.Validator`.