Add support for undefined values

[bintoro]

Basics

As before, objects are populated by default.

class M(Model):
    intfield = IntType()
    stringfield = StringType()
    defaultfield = StringType(default='foo')

>>> m = M()
>>> m.to_primitive()
{'intfield': None, 'stringfield': None, 'defaultfield': 'foo'}

The del statement, which used to be completely broken, can now be used to clear an individual value.

>>> del m.intfield
>>> m.to_primitive()
{'defaultfield': 'foo', 'stringfield': None}

Internally, we have the Undefined singleton as a placeholder to indicate the absence of a value.

>>> m._data
{'intfield': Undefined, 'stringfield': None, 'defaultfield': 'foo'}
>>> 'intfield' in m
False

(Note: The internals of _data are subject to change and should be considered an implementation detail.)

Attempting to access a nonexistent value through an attribute will raise an error that is a subclass of AttributeError.

There are two new parameters recognized by import_loop:

• init_values: What to do with missing values
  • True: initialize to None
  • False: leave as Undefined (default)
• apply_defaults: What to do with fields' defaults
  • True: honor them
  • False: ignore them (default)

Normally, Model.__init__() overrides both of these. Its arguments include init=True, which is just a shortcut for setting both init_values and apply_defaults at once when creating a new instance.

So, to avoid pre-populating the model, one can specify init=False:

>>> m = M({'intfield': 1}, init=False)
>>> m.to_primitive()
{'intfield': 1}

Obviously, init_values and apply_defaults can also be set independently.

Field defaults

Presently, Schematics turns None values into the fields' respective defaults on every import and even during validation, which IMO is plainly a bug.

As of this PR, these issues are fixed by virtue of

• substituting defaults for missing values only
  (i.e., None in the input is now considered an explicit value that will not be replaced)
• having apply_defaults turned off by default (as it's mainly useful during __init__ only).

Controlling what to export

There's a new option export_level that can appear in field definitions and model options. It supersedes serialize_when_none.

export_level	Undefined	None	Empty containers	Other values
ALL	YES	YES	YES	YES
DEFAULT	NO	YES	YES	YES
NOT_NONE	NO	NO	YES	YES
NONEMPTY	NO	NO	NO	YES
DROP	NO	NO	NO	NO

The default level is inherited from the base class options.

serialize_when_none remains as a (deprecated) synonym for export_level: True => DEFAULT, False => NONEMPTY.

export_level can also be passed as an argument to export methods, in which case it will override any field- and model-level settings. In this capacity, it replaces print_none, which is removed.

Missing values appear as None when exported.

>>> m.to_primitive()
{'intfield': 1}
>>> m.to_primitive(export_level=ALL)
{'intfield': 1, 'stringfield': None, 'defaultfield': None}

The model interface

keys(), values(), items(), and __len__() exclude unset fields.

By contrast, atoms() exposes Undefined placeholder objects, as its purpose is to iterate over all fields and serializables.

Compatibility

Existing tests continue to pass unchanged except for

• one test looking at the del statement, which only recently got fixed anyway
• two tests that were passing the now-defunct print_none parameter to export_loop.

Breaking changes are mainly about fixing weird behaviors, most notably the habit of default values to reappear at every turn.