New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seeing "TypeError: exceptions must derive from BaseException" when looking at validator.errors on failed validation #278

Closed
ThatsGobbles opened this Issue Oct 28, 2016 · 10 comments

Comments

Projects
None yet
4 participants
@ThatsGobbles
Copy link

ThatsGobbles commented Oct 28, 2016

This issue tracker is dedicated to bugs and feature requests.

Stack Overflow is the go-to place for general questions and general how-tos. On
SO, questions tagged with the 'cerberus' tag are actively monitored by the
project author, contributors and users.

When reporting a bug, please post an example and make it as simple as possible.
From your use-case throw everything out that is irrelevant for your question
/ isn't necessary to evoke the bug.

Try to adhere as much as possible to the template below:

Used Cerberus version / latest commit: ...

  • I consulted these documentations:
  • I consulted these sections of the docs (add more lines as necessary):
    • ...
    • ...
  • I found nothing relevant to my problem in the docs.
  • I found the documentation not helpful to my problem.
  • I have the capacity to improve the docs when my problem is solved.
  • I have the capacity to submit a patch when a bug is identified.

Use-case abstract

I am using a 'oneof_schema' rule in order to allow either:

  • an empty dictionary, or
  • a dictionary of a specific format

Support request / Bug report

Using: Cerberus 1.0.1 and Python 3.5.2

I'm seeing a strange error when trying to use a 'oneof_schema' rule for my document. The goal is to either allow the 'measurement' field to be an empty dict, or following a specific format.

Examples of documents that (should) pass:

{
    'tokens': [
        {
            'measurement': {
                'qty': { 'num': 10, 'den': 1, },
                'units': 'lbs',
            },
        },
    ],
}

{
    'tokens': [
        {
            'measurement': {
                'qty': 100,
                'units': 'lbs',
            },
        },
    ],
}

{
    'tokens': [
        {
            'measurement': {},
        },
    ],
}

Example of a document that should fail:

{
    'tokens': [
        {
            'measurement': {
                'qty':     { 'num': 'THIS WILL FAIL!', 'den': 1, },
                'units':    'lbs',
            },
        },
    ],
}

Interestingly, when I tested the above "should-fail" document, I get a strange exception:

$ python failure.py
Traceback (most recent call last):
  File "main.py", line 64, in <module>
    print(v.errors)
  File "/home/cheffu/venvs/cerberus-cheffu-test/lib/python3.5/site-packages/cerberus/validator.py", line 360, in errors
    return self.error_handler(self._errors)
  File "/home/cheffu/venvs/cerberus-cheffu-test/lib/python3.5/site-packages/cerberus/errors.py", line 443, in __call__
    self.extend(errors)
  File "/home/cheffu/venvs/cerberus-cheffu-test/lib/python3.5/site-packages/cerberus/errors.py", line 372, in extend
    self.add(error)
  File "/home/cheffu/venvs/cerberus-cheffu-test/lib/python3.5/site-packages/cerberus/errors.py", line 453, in add
    self.insert_group_error(error)
  File "/home/cheffu/venvs/cerberus-cheffu-test/lib/python3.5/site-packages/cerberus/errors.py", line 496, in insert_group_error
    self.insert_logic_error(error)
  File "/home/cheffu/venvs/cerberus-cheffu-test/lib/python3.5/site-packages/cerberus/errors.py", line 516, in insert_logic_error
    raise NotImplemented
TypeError: exceptions must derive from BaseException

If I change the rule to remove the 'oneof_schema' rule (losing the ability to accept an empty dict), I'm able to see the validation error, as expected:

$ python ok.py
{'tokens': [{0: [{'measurement': [{'qty': [{'oneof': ['none or more than one rule validate', {'oneof definition 1': ['must be of integer type'], 'oneof definition 0': ['must be of integer type']}]}]}]}]}]}

The only difference in the code between the two files is the 'oneof_schema' rule in the 'measurement' schema (lines 36-39 in failure.py). Is this an error on my part, or something more insiduous?

Attached are the two Python files: failure.py.txt and ok.py.txt.

@funkyfuture

This comment has been minimized.

Copy link
Member

funkyfuture commented Oct 30, 2016

well, it is simply NotImplemented. :-/

if you would amend that, this would be great. i think the main nut to crack here is to figure out a reasonable way to represent such nested errors to human beings.

@ThatsGobbles

This comment has been minimized.

Copy link

ThatsGobbles commented Oct 31, 2016

Looking at master, it seems that this had already been amended: I'm seeing NotImplementedError being raised as expected, instead of NotImplemented. Guessing this will be rolled into the next release?

It seems that my approach of trying to allow an empty dict was raising an error, but the error wasn't implemented, but the exception raised as a result of THAT error was incorrect. What a mouthful!

That being said, how would I go about doing this kind of validation?

'oneof_schema': [
    AMOUNT_AND_UNITS_SCHEMA,
    {},
],
@funkyfuture

This comment has been minimized.

Copy link
Member

funkyfuture commented Oct 31, 2016

@ThatsGobbles

This comment has been minimized.

Copy link

ThatsGobbles commented Nov 1, 2016

I'll give it a shot and see if I can make a contribution to add that functionality.

@ThatsGobbles

This comment has been minimized.

Copy link

ThatsGobbles commented Nov 2, 2016

Since this is new functionality, I'm not sure what the desired output error dictionary should look like.

schema =    {
                'abc': {
                    'type': 'dict',
                    'oneof_schema': [
                        {
                            'foo': {
                                'type': 'dict',
                                'schema': {
                                    'bar': {
                                        'oneof_type': [ 'integer', 'float' ],
                                    },
                                },
                            },
                        },
                        { 'baz': { 'type': 'string' } },
                    ],
                },
            }
document = { 'abc': { 'foo': { 'bar': 'bad' } } }

If I do the naive thing and replace the NotImplementedErrors with the corresponding insert_*_errors, I get a error tree that looks incorrect:

{'abc': [{'foo': [{'bar': [{'oneof': ['none or more than one rule validate',
                                      {'oneof definition 0': ['must be of '
                                                              'integer type'],
                                       'oneof definition 1': ['must be of '
                                                              'float '
                                                              'type']}]}]}],
          'oneof': ['none or more than one rule validate',
                    {'oneof definition 1': ['unknown field']}]}]}

I would imagine there should be an error message for the outer level 'oneof definition 0', but there is none. Perhaps I'm not understanding the code as well as I need to be.

@funkyfuture

This comment has been minimized.

Copy link
Member

funkyfuture commented Nov 5, 2016

i've done an annotated version of your result:

{'abc':  # 1st level document key node
    [  # container for all messages (there are none) and subnodes of 'abc'
        {'foo':  # 2nd level document key subnode
            [  # container for all messages (there are none) and subnodes of 'foo'
                {'bar':  # 3rd level document key subnode
                    [  # container for all messages (there are none) and subnodes of 'bar'
                        {'oneof':  # subnode for 'oneof'-related details (relating to 'bar')
                            [  # container for the 'oneof' details
                                'none or more than one rule validate',  # overall description
                                 {'oneof definition 0':  # subnode for 'oneof' definition #0
                                     ['must be of integer type'],  # message
                                  'oneof definition 1':
                                     ['must be of float type']}
                            ]
                        }
                    ]
                }
            ],
         'oneof':  # subnode for 'oneof'-related details (relating to 'abc')
            [  # container for the 'oneof' details
                'none or more than one rule validate',  # overall desciption
                {'oneof definition 1':  # subnode for 'oneof' definition #1
                    ['unknown field']  # message / should contain 'baz'
                }
            ]
        }
    ]
}

if think the dict staring in line 3 should be contained here: errors['abc'][-1]['oneof'][-1]['oneof definition 0'].

like so:

{'abc':  # 1st level document key node
    [  # container for all messages (there are none) and subnodes of 'abc'
        {
         'oneof':  # subnode for 'oneof'-related details (relating to 'abc')
            [  # container for the 'oneof' details
                'none or more than one rule validate',  # overall desciption
                {'oneof definition 0':  # subnode for 'oneof' definition #0 / ** missing above **
                    [
                        {'foo':  # 2nd level document key subnode
                            [  # container for all messages (there are none) and subnodes of 'foo'
                                {'bar':  # 3rd level document key subnode
                                    [  # container for all messages (there are none) and subnodes of 'bar'
                                        {'oneof':  # subnode for 'oneof'-related details (relating to 'bar')
                                            [  # container for the 'oneof' details
                                                'none or more than one rule validate',  # overall description
                                                 {'oneof definition 0':  # subnode for 'oneof' definition #0
                                                     ['must be of integer type'],  # message
                                                  'oneof definition 1':
                                                     ['must be of float type']}
                                            ]
                                         }
                                    ]
                                 }
                            ]
                         }
                    ],
                 'oneof definition 1':  # subnode for 'oneof' definition #1
                    [
                        {'bar':  # ** missing above **
                            ['unknown field']  # message
                         }
                    ]

                 }
            ]
        }
    ]
}

does this make sense?

the missing bar reference is an issue of its own.

for testing purposes it would be better to replace one of the oneof with an anyof or allof.

@bryanwweber

This comment has been minimized.

Copy link
Contributor

bryanwweber commented Nov 28, 2016

I'm having the same issue. Has there been any progress on this issue, or should I try to work on a fix? (and interestingly, my application is very similar, to validate quantities with units and uncertainties...)

@bryanwweber bryanwweber referenced this issue Nov 28, 2016

Merged

Add uncertainty #30

5 of 5 tasks complete
@funkyfuture

This comment has been minimized.

Copy link
Member

funkyfuture commented Dec 6, 2016

@bryanwweber it would be great to see this fixed with the next release.

@bryanwweber

This comment has been minimized.

Copy link
Contributor

bryanwweber commented Dec 6, 2016

@funkyfuture When is the next release planned? I will have time to work on this starting in about 1.5 weeks, I think

@nicolaiarocci

This comment has been minimized.

Copy link
Member

nicolaiarocci commented Dec 7, 2016

I won't be releasing anything until January I'm afraid. That should allow plenty of time for sneaking your fix in :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment