New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Class variables? #184

Closed
dsully opened this Issue May 28, 2018 · 18 comments

Comments

7 participants
@dsully
Copy link

dsully commented May 28, 2018

Is it possible to have class variables that are not part of the schema that is parsed by pydantic?

ie:

class MyModel(BaseModel):

     # This is part of pydantic
     group: str = None

     # This is not.
     ACTIONS = ['a', 'b', 'c']```
@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented May 28, 2018

if not var_name.startswith('_') and not isinstance(value, TYPE_BLACKLIST):

Start the name with _ or use a function with the @property decorator.

@dsully

This comment has been minimized.

Copy link

dsully commented May 29, 2018

Thanks. Would you accept a PR that also ignored all uppercase class variable names?

I didn't see this in the documentation, perhaps I missed it?

@samuelcolvin samuelcolvin reopened this May 29, 2018

@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented May 29, 2018

Probably not excluding all uppercase variables, but I would accept a PR which either:

  • exclude_uppercase: bool which when True caused uppercase variables to be excluded from validation
  • exclude_vars: Set[str] where you could provide a bunch of variables names which you wished to exclude
@jaheba

This comment has been minimized.

Copy link
Contributor

jaheba commented Jun 8, 2018

Being able to define attributes which are not being part of the schema being part is also something I'm looking for.

However, I have the suspicion that doing so based on the case of the variable name could be too inflexible. Thus, I would prefer the exclude_vars approach much more.

But how about we add a new type, which marks variables as internal?

class MyModel(BaseModel):
     # This is part of pydantic
     group: str = None

     # This is not.
     ACTIONS: Internal[List[str] = ['a', 'b', 'c']```
@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented Jun 8, 2018

Internal makes a lot of sense to me.

I guess it might upset mypy, but that doesn't both me much.

@jaheba

This comment has been minimized.

Copy link
Contributor

jaheba commented Jun 8, 2018

I think I've found a way to trick mypy. We can abuse a Union, where one case is an internal field. That way, we can check whether that var is internal and still have the mypy guarantees.

class _internal:
    pass

T = TypeVar('T')

Internal = Union[_internal, T]

# accepted by mypy
x: Internal[int] = 42
@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented Jun 8, 2018

Makes sense to me, I neither know nor care whether it's an abuse of typing.

@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented Jun 8, 2018

Humm, maybe we need to think harder about using types to indicate things that don't relate to type. Could be very confusing.

@jaheba

This comment has been minimized.

Copy link
Contributor

jaheba commented Jun 11, 2018

Humm, maybe we need to think harder about using types to indicate things that don't relate to type. Could be very confusing.

I agree.

I think what we are looking for is a way to annotate settings for each field. The problem is that we are limited in ways to do so:

a) variable name (e.g. variables starting with an `_` are currently ignored)
b) type annotation
c) default value the field is set to
d) using external annotations, such as the `Config` class

In Rust for example, one can also annotate things using the #[...] syntax. But we don't have that option in Python.

Out of the ones listed, I would currently prefer c) :

class MyModel(BaseModel):
    x: int = Internal(42)

What I am currently worried about, if there are other annotations we would like to add which would make this approach unfeasable. Maybe something like this would be better than:

class MyModel(BaseModel):
    x: int = Field(42, internal=True, allow_mutation=False)
@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented Jun 11, 2018

That's already work going on. See #190, #129.

Problem is that internal, as it works now couldn't be used with anything else, eg. allow_mutaion=False. Because the variables are just ignored by pydantic.

Also, internal is a confusing name as it indicates they can only be used inside the class. Maybe it should be called "ignored". And I'd rather just keep it in config for cleanness.

@jaheba

This comment has been minimized.

Copy link
Contributor

jaheba commented Jun 11, 2018

That's already work going on. See #190, #129.

Oh, that looks nice. I think that would probably be the right place to add the feature then.

Also, internal is a confusing name as it indicates they can only be used inside the class. Maybe it should be called "ignored". And I'd rather just keep it in config for cleanness.

Agreed internal is just what first came to mind. Although I'm not sure if ignored is unambigious either. But I guess that's a discussion for another time.

@koliyo

This comment has been minimized.

Copy link

koliyo commented Sep 4, 2018

First of all, I would also like this feature.

As a workaround I am trying to use the _foo declaration syntax. But with this I am not allowed to explicitly set the value of the object.

o = Foo(**data)
o._bar = 3

Then I get

ValueError: "Foo" object has no field "_bar"

I just want to exclude it from the external data parameterization.

@Michael-J-Ward

This comment has been minimized.

Copy link

Michael-J-Ward commented Nov 1, 2018

Would taking advantage of typing.ClassVar be a good candidate for this? Seems like pydantic doesn't currently allow those.

from typing import ClassVar
from pydantic import BaseModel

class MyModel(BaseModel):
    foo: ClassVar[int] = 3
    bar: float

Traceback (most recent call last):
  File "python3.6/site-packages/pydantic/validators.py", line 262, in find_validators
    if issubclass(type_, val_type):
TypeError: issubclass() arg 1 must be a class
@Midnighter

This comment has been minimized.

Copy link

Midnighter commented Dec 16, 2018

Just stumbled over this error. I would love to see support for typing.ClassVar.

@tiangolo

This comment has been minimized.

Copy link
Contributor

tiangolo commented Dec 16, 2018

As a workaround:

You can always create a class inside of the model class.

And store there any class-level constants you need.

Pydantic won't interact with it:

from pydantic import BaseModel

class User(BaseModel):
    username: str
   
    class Meta:
        model_type = "user"

user = User(username="john")

print(user)
print(user.dict())
print(user.Meta.model_type)

prints:

User username='john'
{'username': 'john'}
user
@Midnighter

This comment has been minimized.

Copy link

Midnighter commented Dec 16, 2018

That's a fair workaround. I just had some dataclasses already defined and then got bitten by two things:

  1. Can't nest dataclasses using pydantic (yet) #273.
  2. Can't use typing.ClassVar (this issue).
@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented Dec 16, 2018

@tiangolo's suggestion is definitely the most sensible approach and will make sense to anyone coming from django or similar.

However we should definitely implement ClassVar.

@samuelcolvin

This comment has been minimized.

Copy link
Owner

samuelcolvin commented Dec 16, 2018

Anyone wanting to contribute, ClassVar shouldn't be too hard to implement. Help much appreciated.

samuelcolvin added a commit that referenced this issue Dec 27, 2018

@samuelcolvin samuelcolvin referenced this issue Dec 27, 2018

Merged

support ClassVar, #339

3 of 5 tasks complete

samuelcolvin added a commit that referenced this issue Dec 27, 2018

samuelcolvin added a commit that referenced this issue Dec 27, 2018

support ClassVar, (#339)
* support ClassVar, fix #184

* fix tests, update history
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment