New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: Allow narrowing ORM fields #511
Comments
Hi, @antonagestam! You can use this notation: score: 'models.Field[int, int]' = models.IntegerField(default=0) Will it work for you? The problem is that you currently can specify wrong generic params: score: 'models.Field[str, str]' = models.IntegerField(default=0) # wrong, but typechecks! |
@sobolevn Cool, I did not know about that! I think I'm running into a variance issue though. I setup a minimal Django project to test with, and testing with this definition just to have less moving parts to think about (also tested with from __future__ import annotations
from django.db import models
class CountryCode(str):
pass
class Member(models.Model):
name = models.CharField(max_length=200)
country: models.Field[CountryCode, CountryCode] = models.CharField(max_length=2) I get:
If I change the definitions of One further issue though, that might be unrelated to this at all is that I don't get type errors passing field values when instantiating the model. I'm assuming this is unrelated because it passes even with this example: from __future__ import annotations
from django.db import models
class Member(models.Model):
value: models.Field[int, int] = models.IntegerField()
m = Member(value="test") # Expected error here |
@antonagestam I've been trying to get to the bottom of some similar issues and I don't think that Say we have a phantom from __future__ import annotations
from django.db import models
class IntMember(models.Model):
value: models.Field[int, int] = models.IntegerField()
class NegativeIntMember(models.Model):
value: models.Field[NegativeInt, NegativeInt] = models.IntegerField() The field should be contravariant when you set it, as def set_value_to_minus_ten(instance: NegativeIntMember):
instance.value = NegativeInt(-10)
def set_value_to_fifteen(instance: IntMember):
instance.value = 15
set_value_to_minus_ten(IntMember(value=7)) # This is fine
set_value_to_minus_ten(NegativeIntMember(value=NegativeInt(-20)) # This is also fine
set_value_to_fifteen(IntMember(value=7)) # This is fine
set_value_to_fifteen(NegativeIntMember(value=NegativeInt(-20)) # This should raise an error but it should be covariant when you get it, as int_member = IntMember(value=7)
negative_member = NegativeIntMember(value=NegativeInt(-20))
int_value: int = int_member.value # This is fine
negative_value: int = negative_member.value # Also fine
negative_value: NegativeInt = negative_member.value # Also fine
int_value: NegativeInt = int_member.value # This should raise an error |
@leamingrad this looks like a valid proposal! Do you have the time / will to submit an experiment PR? |
@leamingrad Ah, I didn't realize they should be reversed to each other. Very nicely explained! 👍 (Sorry for off topic): Feel free to jump in on phantom-types/discussions if you'd feel like sharing how you use phantom types! We've slowly started to use them in a production project at work now, would be nice with feedback from other users too :) |
I created a new PR: #573 It does make it possible to narrow a field's type, but I found no other way to do that than to use |
@antonagestam I guess so, but it should still check that types are defined corretly. Maybe we can incorporate this change into your PR as well? |
@sobolevn Yes, fixing that would be awesome. I started looking into it a bit but I somehow always find it extremely hard to understand what's going on in mypy plugins. I assume changes have to be made in https://github.com/typeddjango/django-stubs/blob/master/mypy_django_plugin/transformers/fields.py, but I would need some help to figure out how get types from explicit arguments when provided. |
These are the results I get when not using Both these give the error below, the first one probably makes sense, but the second one is because the explicit arguments aren't picked up. published: models.Field[Year, Year] = models.IntegerField()
published: models.Field[Year, Year] = models.IntegerField[Year, Year]()
Also tried these: published: models.IntegerField[Year, Year] = models.IntegerField()
published: models.IntegerField[Year, Year] = models.IntegerField[Year, Year]()
This doesn't give an error at assignment, but published = models.IntegerField[Year, Year]()
|
Yes, looks like we need to fix this in our plugin.
Yes, I would start from here: django-stubs/mypy_django_plugin/main.py Lines 208 to 210 in caaa23a
Sure, please feel free to ask! |
I am going to leave this open, since we can also change how our plugin works. |
@antonagestam Thanks for getting a PR merged so quickly! To be honest, I don't actually use phantom types, but it seemed like a clearer way to give examples (most of the issues I have been working with are to do with relations between abstract models which this is a precursor of). That said, it seems like an interesting idea so I will definitely check it out. |
This is pure gold, @antonagestam I am marketing it a lot 🙂 |
Hi, I'm trying to add types to django-phonenumber-field package, and it turned out trickier than expected. I've got it working locally with class PhoneNumberField(models.CharField[_T]): # type: ignore
@overload
def __new__(
cls, *args, null: Literal[False] = ..., **kwargs
) -> PhoneNumberField[PhoneNumber]: ...
@overload
def __new__(
cls, *args, null: Literal[True] = ..., **kwargs
) -> PhoneNumberField[PhoneNumber | None]: ...
@overload
def __new__(cls, *args, **kwargs) -> PhoneNumberField[PhoneNumber]: ...
def __new__(cls, *args, **kwargs) -> PhoneNumberField[PhoneNumber | None]: ... And now i want to make proper PR for the project, but we decided to use mypy and django-stubs, and i cannot figure out how to properly assign types. My first try was to use Then i tried using
Any ideas how to do this? |
I am currently working on implementing phantom types for Python, which, when implemented throughout an application, would allow statically proving that a value saved on model is validated.
To be able to use this with django-stubs there would need to be some way of narrowing the types that ORM fields hold.
One example would be a field for country codes:
This makes mypy error if I try to assign a string value to the field:
Unless I first prove that the string is a valid country code:
Unfortunately mypy raises an error when annotating ORM fields:
Although my usecase is phantom types, I believe a feature that enables this would also work equally well with
typing.NewType
types.This can be worked around with using the usual dirty tricks of ignore comments or using e.g.
typing.cast(CountryCode, CharField())
. But it would be very nice to see builtin support for this. I think it would be optimal if this builtin support also verifies that the annotated type is a subtype of the type that the field holds, so that the type annotated to aCharField
must be a subtype ofstr
and so on.What do you think?
PS.: This is a proposal to get a discussion started, I'm not asking anyone to implement this for me.
The text was updated successfully, but these errors were encountered: