Cerberus tests fail under Python 3.8

[ignatenkobrain]

Used Cerberus version / latest commit: 1.3.1

Bug report / Feature request

I'm maintainer of Cerberus in Fedora. Python folks are trying to rebuild all packages against Python 3.8. However, cerberus one tests fail with 3.8 (but succeed with 3.7). Below you can find output.

______________________________ test_nested_oneofs ______________________________

validator = <cerberus.validator.Validator object at 0x7f6ccf22a0a0>

    def test_nested_oneofs(validator):
>       validator.schema = {
            'abc': {
                'type': 'dict',
                'oneof_schema': [
                    {
                        'foo': {
                            'type': 'dict',
                            'schema': {'bar': {'oneof_type': ['integer', 'float']}},
                        }
                    },
                    {'baz': {'type': 'string'}},
                ],
            }
        }

cerberus/tests/test_validation.py:1522: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
cerberus/validator.py:562: in schema
    self._schema = DefinitionSchema(self, schema)
cerberus/schema.py:82: in __init__
    self.validate(schema)
cerberus/schema.py:259: in validate
    self._validate(schema)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'DefinitionSchema' object has no attribute 'schema'") raised in repr()] DefinitionSchema object at 0x7f6ccf22acd0>
schema = {'abc': {'oneof': [{'schema': {'foo': {'schema': {...}, 'type': 'dict'}}}, {'schema': {'baz': {'type': 'string'}}}], 'type': 'dict'}}

    def _validate(self, schema):
        if isinstance(schema, _str_type):
            schema = self.validator.schema_registry.get(schema, schema)
    
        if schema is None:
            raise SchemaError(errors.SCHEMA_ERROR_MISSING)
    
        schema = copy(schema)
        for field in schema:
            if isinstance(schema[field], _str_type):
                schema[field] = rules_set_registry.get(schema[field], schema[field])
    
        if not self.schema_validator(schema, normalize=False):
>           raise SchemaError(self.schema_validator.errors)
E           cerberus.schema.SchemaError: {'abc': [{'oneof': [{'schema': ['no definitions validate', {'anyof definition 0': [{'foo': [{'schema': ['no definitions validate', {'anyof definition 0': [{'bar': [{'oneof_type': ['unknown rule']}]}], 'anyof definition 1': [{'bar': ['unknown rule']}]}]}]}], 'anyof definition 1': [{'foo': ['unknown rule']}]}]}]}]}

cerberus/schema.py:275: SchemaError

[funkyfuture]

thanks for reporting. a fix has been merged to the '1.3.xbranch and will be available with the1.3.2` release.

btw, we would have loved to continue using https://github.com/ignatenkobrain/sphinxcontrib-issuetracker

[frenzymadness]

I haven't managed to fix it yet and because the code is kinda complicated I am not sure that I do, but I might have some important info.

I've tried to test the behavior with the dictionary from the test and it seems that the representation of the dictionary in the AST is different between 3.7 and 3.8.

The dictionary I'm talking about:

{
    'abc': {
        'type': 'dict',
        'oneof_schema': [
            {
                'foo': {
                    'type': 'dict',
                    'schema': {'bar': {'oneof_type': ['integer', 'float']}},
                }
            },
            {'baz': {'type': 'string'}},
        ],
    }
}

List of nodes in Python 3.7:

<class '_ast.Module'>
<class '_ast.Expr'>
<class '_ast.Dict'>
<class '_ast.Str'>
<class '_ast.Dict'>
<class '_ast.Str'>
<class '_ast.Str'>
<class '_ast.Str'>
<class '_ast.List'>
<class '_ast.Dict'>
<class '_ast.Dict'>
<class '_ast.Load'>
<class '_ast.Str'>
<class '_ast.Dict'>
<class '_ast.Str'>
<class '_ast.Dict'>
<class '_ast.Str'>
<class '_ast.Str'>
<class '_ast.Str'>
<class '_ast.Dict'>
<class '_ast.Str'>
<class '_ast.Str'>
<class '_ast.Str'>
<class '_ast.Dict'>
<class '_ast.Str'>
<class '_ast.List'>
<class '_ast.Str'>
<class '_ast.Str'>
<class '_ast.Load'>

List of nodes in Python 3.8:

<class '_ast.Module'>
<class '_ast.Expr'>
<class '_ast.Dict'>
<class '_ast.Constant'>
<class '_ast.Dict'>
<class '_ast.Constant'>
<class '_ast.Constant'>
<class '_ast.Constant'>
<class '_ast.List'>
<class '_ast.Dict'>
<class '_ast.Dict'>
<class '_ast.Load'>
<class '_ast.Constant'>
<class '_ast.Dict'>
<class '_ast.Constant'>
<class '_ast.Dict'>
<class '_ast.Constant'>
<class '_ast.Constant'>
<class '_ast.Constant'>
<class '_ast.Dict'>
<class '_ast.Constant'>
<class '_ast.Constant'>
<class '_ast.Constant'>
<class '_ast.Dict'>
<class '_ast.Constant'>
<class '_ast.List'>
<class '_ast.Constant'>
<class '_ast.Constant'>
<class '_ast.Load'>

From the documentation, it seems that the Num, Str, Bytes, NameConstant and Ellipsis are considered as deprecated and will be removed. The upstream issue is here and contains some useful info and links to other issues.

I am not saying that this is the cause of the problem here but the different representation of the same thing may mean something.

[frenzymadness]

Well, I should refresh pages more frequently 😄