-
Notifications
You must be signed in to change notification settings - Fork 234
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Covariance or contravariance #2
Comments
The PEP currently says: "As with function arguments, generics are covariant, which is in spirit of duck typing." Jeremy Siek objects to this statement. Jeremy wants to be able to express strictly static checking as an option, doesn't want to limit strength of type system. Put another way, Jeremy's hope for gradual typing is to let programmers choose anywhere in between fully dynamic and fully static checking, but if the type system is weakened by allowing covariance, then the fully static option goes away. |
Currently mypy special cases certain ABCs to support covariant subtyping (such as I don't think that We could allow a type variable to be defined as covariant (or contravariant). For example,
Here A covariant type variable could not generally be used in a method argument type to maintain type safety (I can give an example if it's not obvious why this makes sense). There are some other technicalities which the PEP can probably ignore. |
I happened to mention this to Peter Norvig, and he had the following to say:
He also reminded me that Josh Bloch has said something like "I think we made it too complicated there". |
Jukka's suggestion for a One thing I want to note is that languages like C# and Java get away with covariant arrays only because they perform run-time checks that raise exceptions when ill-typed data is written to arrays. Since Python doesn't have those, covariant arrays could provide a really easy and (to most programmers) unexpected way to write programs with type errors that type analyzers can't detect and that are hard to debug. |
I agree that "in" and "out" are better terms. Since "in" is reserved, we could do something like this: T = typevar('T', kind='in') As a bonus, 'kind' is probably more Googleable than 'in'. Michael, good point about the lack of runtime checking. It's a good argument for the importance of getting rid of glaring type safety holes (when feasible). |
I would suggest to use invariance by default, but to introduce declaration-site variance annotations in order to simplify things for users of generic classes. You can specify variance of a generic parameter inside a) its usage as a type annotation or b) its declaration as a parameter of a generic class. For many classes it makes sense to specify default variance at the declaration site. (Scala allows that while Java doesn't). For example, immutable classes are often covariant and mutable classes are usually invariant. Specifying variance at the declaration site would allow the users of these classes not to bother with variance and use simple type annotations like The example below statically prevents an from abc import abstractmethod
from typing import Var, Generic, Dict
# Clases from typing with variance set at the declaration site
T = Var('T') # kind='inout', invariant
TOut = Var('T', kind='out') # covariant
class Iterable(Generic[TOut]):
@abstractmethod
def __iter__(self) -> 'Iterator[TOut]': pass
class Iterator(Iterable[TOut]):
@abstractmethod
def __next__(self) -> TOut: pass
class List(Generic[T]):
def add(self, item: T): pass
def pop(self) -> T: pass
...
def evaluate_employees(employees: Iterable[Employee]) -> Dict[Employee, float]:
return {x: evaluate(x) for x in employees}
def hire_employee(employees: List[Employee], new_empoloyee: Employee) -> None:
employees.add(new_employee)
def test(managers: List[Manager]):
evaluate_employees(managers) # OK: List[Manager] is a subtype of Iterable[Employee]
john = Engineer('John')
hire_employee(managers, john) # Error: List[Manager] is not a subtype of List[Employee]
assert john in managers Sometimes however the user may want to allow def best_employee(employees: List[Var('T', Employee, kind='out')]) -> Employee:
return max(employees, key=evaluate)
def test(managers: List[Manager]):
best = best_employee(managers) # OK: List[Manager] is a subtype of List[Var('T', Employee, kind='out')] The annotation of What could happen if T = Var('T', kind='out') # covariant
class List(Generic[T]):
def add(self, item: T): pass
def pop(self) -> T: pass
...
def hire_employee(employees: List[Employee], new_empoloyee: Employee) -> None:
employees.add(new_employee)
def test(managers: List[Manager]):
john = Engineer('John')
hire_employee(managers, john) # OK: List[Manager] is a subtype of List[Employee]
assert john in managers # AssertionError: John is in the list of managers now An alternative way to simplify things for the user is to use covariance by default, but to limit kinds of methods that could be called on the value of a covariant type to only T = Var('T', kind='out') # covariant
class List(Generic[T]):
def add(self, item: T): pass
def pop(self) -> T: pass
...
def hire_employee(employees: List[Employee], new_empoloyee: Employee) -> None:
employees.add(new_employee) # Error: Employee is not a subtype of Var('T', Employee, kind='out')
# i.e. only super types of Employee allowed here
def test(managers: List[Manager]):
john = Engineer('John')
hire_employee(managers, john) # OK: List[Manager] is a subtype of List[Employee]
assert john in managers But this approach might be even more complicated since error messages are harder to understand. And it is not clear what you have to change in order to fix your code. Compare:
|
I like declaring the variance kind with the type variable, but I also want On Thu, Nov 13, 2014 at 10:11 AM, Andrey Vlasovskikh <
--Guido van Rossum (python.org/~guido) |
@gvanrossum Exactly. I've compared a covariant TOut = Var('T', kind='out') # covariant
class Sequence(Generic[TOut]):
@abstractmethod
def __getitem__(self, index: int) -> TOut]: pass |
@vlasovskikh - I can't completely follow your long message -- either you are basically reiterating the need for mutable containers to use invariance, or your point is too subtle (or longwinded) for me. Regarding the use of |
@gvanrossum I want to stress two points in my message:
The rest of my message is just an explanation why I feel that those two are important. I didn't see any mentions of declaration-site variance in discussions of typing, so felt that it was necessary to remind about this option in this discussion. Anything else including |
OK, I like that generic types should default to invariant. (However, Tuple and Callable have different rules -- see the theory article, which is now https://www.python.org/dev/peps/pep-0483/. Tuple. being immutable, is covariant -- Callable is covariant in the return type, but contravariant in the arguments. I doubt this is controversial.) For now I'll take your word on declaration-site variance vs. use-site variance; once I think I understand it I'll let you know what I think of it. :-) |
@gvanrossum That makes perfect sense for tuple and callable types. |
I found a good discussion of declaration-site variance on Wikipedia: http://en.wikipedia.org/wiki/Covariance_and_contravariance_%28computer_science%29#Declaration-site_variance_annotations Let me summarize it in my own words:
If we choose declaration-site variance, there is an opportunity here for a "cute" notation borrowed from Scala and OCaml: +T for using T covariantly, -T for using it contravariantly (C# uses out/in, but the latter is a bit problematic because it is a Python keyword). This is syntactically valid Python, and easy to implement in the runtime typing.py that I am developing, by overloading
(TBD: Add usage example with Employee/Manager, check with vlasovskikh's comment.) |
OK, I think I've got it right, and your Using my "cute" spelling idea, perhaps we could replace your
with
(This goes against the rule that a I'm not sure what Jukka thinks, and we'll have to see how this works out in a larger practice trial (i.e. Python 3.5 :-) but so far I like it. Apart from how to spell co/contravariant type variables (and whether that should be part of the variable declaration or part of the class definition) I also think we need to discuss bounded type variables in general, but I think that can be done separate from the co/contravariant discussion. (Maybe in #1.) |
@gvanrossum That's right. The I should note that declaration-site variance and use-site variance can co-exist in a language. Maybe in order to simplify things we could use only declaration-site variance. Here is an example of declaration-site variance vs use-site variance. Given a hierarchy of
T = TypeVar('T'): ...
class Iterable(Generic[+T]): ...
def best_employee(employees: Iterable[Employee]) -> Employee:
return max(employees, key=evaluate)
E = TypeVar('E', Employee)
def best_employee(employees: List[+E]) -> E:
return max(employees, key=evaluate) In this case having covariant |
OK, that's very clear, and I am beginning to understand what the 39% quoted at http://en.wikipedia.org/wiki/Covariance_and_contravariance_%28computer_science%29#Comparing_Declaration-site_and_Use-site_annotations refers to. I also now understand what they mean with "the Scala Collections library defines three separate interfaces for classes which employ covariance". I'd love some input from Jukka on this issue (summary: whether to support only declaration-site variance declarations (first bullet), or also use-site variance declarations (second bullet); and what you think of the +T/-T syntax instead of (or as a shorthand for) kind='out'/'in' in the TypeVar() call. Also, what "e = TypeVar('E', Employee)" should mean (without the variance declaration). |
My vote goes to only having declaration-site variance. Many uses of wildcard types as in Java programs (use-site variance) could probably be replaced by using just type variables or using
We can use a type variable to achieve a similar effect:
We could also rewrite it using
I looked at this some years ago and it seemed that wildcard types were rarely used in interesting ways, but they may be more widely used in recent Java code. |
SGTM! On Wednesday, January 14, 2015, Jukka Lehtosalo notifications@github.com
--Guido van Rossum (on iPad) |
@JukkaL Your second example is not equivalent to your first example if we consider |
This is now the most important issue left to decide. It's pretty complex (I always get a headache thinking about this stuff) and I don't really like the |
I wrote something up here: https://github.com/ambv/typehinting/blob/master/VARIANCE.rst |
FWIW, I think I got in/out mixed up. According to the wikipedia covariant is 'out' (or +) and contravariant is 'in' (or -). This is because I wrote both the typing.py patch and the VARIANCE.rst file from memory. So much for in/out being mnemonic. :-( |
I can't sleep. :-) I fixed this in VARIANCE.rst and typing.py. But my real proposal is to drop |
@gvanrossum That sounds reasonable -- it has the benefit that anybody can easily google what the technical terms mean, and even though the terms are technical, they are pretty widely used with a generally agreed on meaning. |
How would this interact with constraints? IIUC, the current proposal does something like this:
But consider:
Is this legal? Do we need to cast |
We now have agreement on this (we're going to use covariant=True/contravariant=True) and typing.py already implements this, but we still need to update the PEP. @NYKevin: This is not what constraint arguments to type variables are meant for. You should just use Employee instead of a type variable. |
As Lukasz mentions in NOTES-ambv.txt: "The covariance/contravariance discussion needs to happen at some point."
Reference: http://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)
I haven't fully digested all the viewpoints yet. It seems that mypy stays on the safe side and treats e.g. Set[Employee] and Set[Manager] as distinct types. However, Set[Employee] is acceptable where a Set (without parameter) is expected.
The text was updated successfully, but these errors were encountered: