New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support NewType #115

Closed
TimSimpsonR opened this Issue Jan 9, 2018 · 4 comments

Comments

2 participants
@TimSimpsonR

TimSimpsonR commented Jan 9, 2018

Pydantic does not seem to support NewType.

Given this code:

import typing as t
import pydantic

BossId = t.NewType('BossId', str)
EmployeeId = t.NewType('EmployeeId', str)
CustomerId = t.NewType('CustomerId', str)

class DirectReport(pydantic.BaseModel):
    boss: BossId
    employee: EmployeeId

The code will hit an error as it's being parsed:

my_code.py:8: in <module>
    class DirectReport(pydantic.BaseModel):
.tox/py36/lib/python3.6/site-packages/pydantic/main.py:82: in __new__
    config=config,
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:86: in infer
    description=field_config and field_config.get('description'),
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:72: in __init__
    self._prepare(class_validators)
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:111: in _prepare
    self._populate_validators(class_validators)
.tox/py36/lib/python3.6/site-packages/pydantic/fields.py:190: in _populate_validators
    *(get_validators() if get_validators else find_validators(self.type_)),
.tox/py36/lib/python3.6/site-packages/pydantic/validators.py:156: in find_validators
    raise TypeError(f'error checking inheritance of {type_!r} (type: {display_as_type(type_)})') from e
E   TypeError: error checking inheritance of <function NewType.<locals>.new_type at 0x407b26388> (type: function)

I currently am using Pydantic with this work around which involves monkey-patching:

import types
import pydantic

def find_validators(type_):
    if type_ is t.Any:
        return []
    for val_type, validators in pydantic.validators._VALIDATORS:
        try:
            if issubclass(type_, val_type):
                return validators
        except TypeError as e:
            if isinstance(type_, types.FunctionType):
                super_type = getattr(type_, '__supertype__')
                if super_type:
                    # Assume this is a NewType
                    if super_type != type_:
                        return find_validators(super_type)
            raise TypeError('error checking inheritance of '
                            f'{type_!r} (type: '
                            f'{pydantic.validators.display_as_type(type_)})'
                            ) from e
    raise pydantic.validators.ConfigError(f'no validator found for {type_}')


# Alters behavior of Pydantic so NewType will work (!!)
pydantic.validators.find_validators = find_validators
pydantic.fields.find_validators = find_validators

@TimSimpsonR TimSimpsonR changed the title from Pydantic should support NewType to support NewType Jan 9, 2018

@samuelcolvin

This comment has been minimized.

Owner

samuelcolvin commented Jan 10, 2018

Surely simpler to construct the class properly:

class BossId(str):
    pass

That way you can easily add custom validation for BossIds etc.

If you really don't want to do that, why not use:

BossId = type('BossId', str, {})

?

If for some reason this really doesn't work, I'd be happy to accept a pull request for your fix above. But we might also need extra logic so that DirectReport.boss is an instance of BossId not just a str. How would this work?

@TimSimpsonR

This comment has been minimized.

TimSimpsonR commented Jan 10, 2018

Hmm, good point that class Type(str) could be a valid work around. I might consider that for string types that require validation or if I need the type info to be attached to the value at runtime.

NewType is designed to be cheap; while MyPy validates it, at runtime it's only a function call, and the instances it creates are the underlying type. So I think it would be ok if Pydantic treated BossId as just a str too.

@samuelcolvin

This comment has been minimized.

Owner

samuelcolvin commented Jan 13, 2018

humm, ok but I bet you someone in future will come back and whinge when they get confused about the type of DirectReport.boss.

Since your change above should have no effect on performance for other usage I'd be happy to accept a pull request for this change provided it has full coverage and documentation.

@samuelcolvin

This comment has been minimized.

Owner

samuelcolvin commented Jun 20, 2018

Happy to accept a PR which supports this provided it's not to complicated.

CustomerId should be validated with the standard str validation logic.

Gr1N added a commit to Gr1N/pydantic that referenced this issue Jul 3, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment