Cannot use OptionalRecurse with a default of None

[jwodder]

Consider the following code:

import json
import cfgv

CONFIG_SCHEMA = cfgv.Map(
    "Config", None,
    cfgv.RequiredRecurse(
        "foo",
        cfgv.Map("foo", None, cfgv.Required("bar", cfgv.check_int)),
    ),
    cfgv.OptionalRecurse(
        "gnusto",
        cfgv.Map("gnusto", None, cfgv.Required("cleesh", cfgv.check_int)),
        None,
    ),
)

print(cfgv.load_from_filename(
    "cfgv-bug.json",
    schema=CONFIG_SCHEMA,
    load_strategy=json.loads,
))

cfgv-bug.json:

{
    "foo": {"bar": 42}
}

Running the above program with cfgv 3.2.0 on Python 3.9.1 produces the following error:

Traceback (most recent call last):
  File "/Users/jwodder/work/dev/tmp/cfgv-bug.py", line 17, in <module>
    print(cfgv.load_from_filename(
  File "/Users/jwodder/work/dev/tmp/venv/lib/python3.9/site-packages/cfgv.py", line 411, in load_from_filename
    return apply_defaults(data, schema)
  File "/Users/jwodder/work/dev/tmp/venv/lib/python3.9/site-packages/cfgv.py", line 381, in apply_defaults
    return schema.apply_defaults(v)
  File "/Users/jwodder/work/dev/tmp/venv/lib/python3.9/site-packages/cfgv.py", line 249, in apply_defaults
    item.apply_default(ret)
  File "/Users/jwodder/work/dev/tmp/venv/lib/python3.9/site-packages/cfgv.py", line 94, in _apply_default_optional_recurse
    _apply_default_required_recurse(self, dct)
  File "/Users/jwodder/work/dev/tmp/venv/lib/python3.9/site-packages/cfgv.py", line 84, in _apply_default_required_recurse
    dct[self.key] = apply_defaults(dct[self.key], self.schema)
  File "/Users/jwodder/work/dev/tmp/venv/lib/python3.9/site-packages/cfgv.py", line 381, in apply_defaults
    return schema.apply_defaults(v)
  File "/Users/jwodder/work/dev/tmp/venv/lib/python3.9/site-packages/cfgv.py", line 247, in apply_defaults
    ret = v.copy()
AttributeError: 'NoneType' object has no attribute 'copy'

My goal is to create a field that can either be set to a Map or omitted entirely. Since there's no OptionalRecurseNoDefault, I thought that setting the field's default to None would work instead. Is the recommended way to get what I want to just set the default to an empty dict, or is there a better strategy?

[asottile]

yeah I think the idea is to use default={}

most of the point of this library and the defaulting is to make it so you don't have to check for presence after applying defaults -- seems odd that you would want to preserve an optional mapping as None instead of the default values for that map

[jwodder]

@asottile In my case, the field being absent has a distinct meaning (of "do not do this thing") which differs from the field having its default values (which means "do this thing with the defaults"), which is why I want to check for field presence in the first place.

[asottile]

you could write your own OptionalNoDefaultRecurse

something like

def _apply_default(self, dct):
    if self.key in dct:
        return OptionalRecurse.apply_default(self, dct)

def _remove_default(self, dct):
    if self.key in dct:
        return OptionalRecurse.remove_default(self, dct)

OptionalNoDefaultRecurse = collections.namedtuple('OptionalNoDefaultRecurse', ('key', 'schema'))
OptionalNoDefaultRecurse.check = OptionalRecurse.check
OptionalNoDefaultRecurse.check_fn = OptionalRecurse.check_fn
OptionalNoDefaultRecurse.apply_default = _apply_default
OptionalNoDefaultRecurse.remove_default = _remove_default

typed into github so likely there'll be a few things to work out

alternatively, your consuming code could treat empty dictionary differently (Required fields won't set their defaults)