Pydantic Model that also inherits from Exception

[aaronkmiller]

Question

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.6.1
            pydantic compiled: True
                 install path: /home/aaronm/pydantic_test/.venv/lib/python3.8/site-packages/pydantic
               python version: 3.8.5 (default, Jul 27 2020, 08:42:51)  [GCC 10.1.0]
                     platform: Linux-5.8.3-arch1-1-x86_64-with-glibc2.2.5
     optional deps. installed: []

If I create a class that inherits from pydantic.BaseModel and Exception:

from pydantic import BaseModel

class CustomException(BaseModel, Exception):
	some_field: str

e = CustomException(some_field='some_value')

I get a lay-out conflict:

Traceback (most recent call last):
  File "custom_exception.py", line 3, in <module>
    class CustomException(BaseModel, Exception):
  File "pydantic/main.py", line 309, in pydantic.main.ModelMetaclass.__new__
  File "/usr/lib/python3.8/abc.py", line 85, in __new__
    cls = super().__new__(mcls, name, bases, namespace, **kwargs)
TypeError: multiple bases have instance lay-out conflict

I think because BaseException and ABCMeta both define __new__?

My use case is that I'm writing a Python client for a REST API using Pydantic. That API provides a JSON schema for all of its possible responses, including HTTP exceptions like 422 and 424.

While I don't care about validating the exceptions against the JSON schema, Pydantic has a number of features I'd like to use when parsing them: alias_generator to convert from the API's camelCase to snake_case, converting strings to enums and code completion in PyCharm. And since I'm using Pydantic models for API requests and non-exception responses, I'd prefer to handle the exceptions in a similar manner for consistency.

But I still need to raise the exception responses, and they should be instances of requests.exceptions.HTTPError.

It seems like this isn't possible with the current version of Pydantic?

[huonw]

We'd love something here too.

We're really enjoying pydantic for defining our API models and converting to JSON in a lambda function. A sketch of the relevant part of what we wish our code could look like is:

import abc
from pydantic import BaseModel

class Response(BaseModel, abc.ABC):
    @abc.abstractmethod
    def status_code(self) -> int:
        ...

class ErrorResponse(Response, Exception):
    message: str

class NotFoundResponse(ErrorResponse):
    def status_code(self) -> int:
        return 404
...

def handler(...):
    try:
        with database_session() as session:
            if some_condition:
                raise NotFoundResponse(message="...")

            response = ... # some valid response
    except ErrorResponse as e:
        response = e

    return {"statusCode": response.status_code(), "body": response.json()}

Using exceptions for our responses makes the control flow simpler, and also means, for instance, the database_session context manager can automatically rollback any transactions.

Unfortunately, the code above doesn't work, so we instead have to introduce an extra ErrorResponseError class (NEW/CHANGED comments to highlight differences):

class ErrorResponse(Response): # CHANGED: no Exception base class
    message: str

class ErrorResponseError(Exception): # NEW
    def __init__(self, response: ErrorResponse):
        self.response = response

...

def handler(...):
    try:
        with database_session() as session:
            if some_condition:
                raise ErrorResponseError(NotFoundResponse(message="...")) # CHANGED: extra wrapper

            response = ... # some valid response
    except ErrorResponseError as e:
        response = e.response # CHANGED: extra property access

    return {"statusCode": response.status_code(), "body": response.json()}

Obviously this isn't particularly bad, and we're happy with it, but it'd be very slightly nicer if we could remove the ErrorResponseError intermediary.

[samuelcolvin]

Humm, without looking into this I think it might be non-trivial to support.

But if your can dig around in our __new__ function and find a simple way, I'd be happy to review a pr.

[mariusvniekerk]

So i took a brief look at this, it seems that if we don't create __slots__ ourselves, everything works

So if we just don't have __slots__ = ('__dict__', '__fields_set__') in BaseModel then everything works (except if you have private fields which get turned into __slots__ atm over here https://github.com/samuelcolvin/pydantic/blob/master/pydantic/main.py#L320

[mariusvniekerk]

For example, whilst obviously a bad idea, the following hack does work to illustrate whats going on.

from pydantic import BaseModel, Field
from pydantic.main import ModelMetaclass

# clear slots and the automatically generated slot descriptors
d = dict(BaseModel.__dict__)
old_slots = d.pop('__slots__', None)
for k in old_slots:
    del d[k]
SlotlessBaseModel = ModelMetaclass('SlotlessBaseModel', BaseModel.__bases__, d)


class Error(SlotlessBaseModel, Exception):
    message: str = Field()


e = Error(message="foo")
print(e.message)
raise e

[samuelcolvin]

Thanks for using pydantic. 🙏

As part of a migration to using discussions and cleanup old issues, I'm closing all open issues with the "question" label. 💭 🆘 🚁

I hope you've now found an answer to your question. If you haven't, feel free to start a 👉 question discussion.

[huonw]

Hi @samuelcolvin, great to see questions moving to a more appropriate tool. 👍

For this particular one, it seems like it started out as a question, but may have become a feature request given the answer seems to be 'not yet possible' (or at least, only possible via hacks) and so may be more appropriate to remain as an issue? Up to you, though!

[samuelcolvin]

Feature requests are also using discussions until they're agreed.

I'm not going to look into this myself, just creating a feature request isn't going to help. Someone needs to sit down and work out what's viable, then it might be worth going further.

[huonw]

Thanks, makes total sense! I hadn't noticed that extra detail about, since it seems like it's just under the Issue > New path.

I'd looked at https://pydantic-docs.helpmanual.io/contributing/ and didn't see a specific discussion of this new policy, so I opened #2371 to hint at it. All good if that's not quite right (and thanks all for tolerating my noise here).

[ghost]

Even though this thread is closed I though I might drop my personal solution to this problem for anyone who may find this useful. To solve the issue of not being able to inherit from both BaseModel and Exception I went with composition instead of inheritance.

with the following quick-any dirty decorator:

class DeserializedError(Exception): pass

def raises(error: Exception):
    def wrap(cls: type):
        
        if not hasattr(cls, '__raise__'):
            raise TypeError(f'Class decorated with `raises` must implement `__raise__`')
        
        class ErrorProxy(error, DeserializedError):
                
            def __init__(self, **data):
                config = cls(**data)
                super().__init__(*config.__raise__())

            @classmethod
            def __get_validators__(cls):
                yield cls._validate
        
            @classmethod
            def _validate(cls, value, **kwargs):
                if isinstance(value, cls):
                    return value
                else:
                    return cls(**value)
            
        original_name = error.__name__

        ErrorProxy.__name__ = 'Deserialized' + error.__name__
        ErrorProxy.__qualname__ = 'Deserialized' + error.__qualname__

        return ErrorProxy
    return wrap

Creating a model

one can define a pydantic compatible model that can be used to raise errors as follows:

@raises(ValueError)
class ValueErrorModel(BaseModel):
    name: Literal['ValueError']
    message: str = ""
    
    def __raise__(self):
        return (self.message, )  # return the arguments that are passed to the exception

Usage

class Foo(BaseModel):
    error: ValueErrorModel
        
foo = Foo(error=dict(name='ValueError', message='Hello world'))

try:
     raise foo.error
except ValueError as error:
    print(error)

[c00kiemon5ter]

a different way of doing this is with a dataclass -- in general I prefer dataclasses over the inheritance with BaseModel. pyright doesn't like it, but validates with mypy:

from pydantic.dataclasses import dataclass


@dataclass
class CustomException(Exception):
    some_field: str


if __name__ == "__main__":
    try:
        raise CustomException("foo")
    except CustomException as e:
        print(f"error occurred: {e.some_field}")

[rafalkrupinski]

Is there a way to also fill Exception.args without breaking pydantic validation?

@pydantic.dataclass
class CustomException(Exception):
    detail: str
    def __init__(self, detail: str) -> None:
        super().__init__(detail)

raise CustomException(detail='OMG')

pydantic.error_wrappers.ValidationError: 1 validation error for CustomException
detail
  field required (type=value_error.missing)