Require default arguments to type check only when not specified at the call site?

[sharmaeklavya2]

Consider this function (It is a simplified version of a function in Zulip).

ItemT = TypeVar('ItemT')
CItemT = TypeVar('CItemT') # compressed Item

# data is low-speed storage. cache is high-speed storage
def fetch(key, data, cache, compress, decompress):
    # type: (str, Mapping[str, ItemT], MutableMapping[str, CItemT], Callable[[ItemT], CItemT], Callable[[CItemT], ItemT]) -> Tuple[ItemT, bool]
    try:
        cvalue = cache[key]
        return (decompress(cvalue), True)
    except KeyError:
        pass
    value = data[key]
    cache[key] = compress(value)
    return (value, False)

This works as expected (see complete example here).

Compression can be useful, but I usually don't require it. So in most cases, CItemT will be the same as ItemT and I'll pass lambda x: x as compress and decompress.

However, if I add default values to compress and decompress like this:
def fetch(key, data, cache, compress=lambda x: x, decompress=lambda x: x):
I get these errors:

cache.py:7: error: Incompatible return value type (got "ItemT", expected "CItemT")
cache.py:7: error: Incompatible types in assignment (expression has type Callable[[ItemT], ItemT], variable has type Callable[[ItemT], CItemT])
cache.py:7: error: Incompatible return value type (got "CItemT", expected "ItemT")
cache.py:7: error: Incompatible types in assignment (expression has type Callable[[CItemT], CItemT], variable has type Callable[[CItemT], ItemT])

I think if I use default values for compress and decompress, mypy should not give errors. It should instead enforce CItemT to be the same as ItemT at the place where I'm calling this function using default arguments for compress and decompress.

This behavior has recently come up. It was not present in mypy 0.4.2.

[gvanrossum]

You should be able to solve this using @overload instead. We don't support that yet but we are planning to: #1136

[rwbarton]

There are two separate issues here.

• The (definitely incorrect) program

  from typing import TypeVar, Callable
  
  T = TypeVar('T')
  S = TypeVar('S')
  
  def f(x: T, y: S) -> None:
      comp = lambda t: t  # type: Callable[[T], S]

  was accepted by mypy 0.4.2 but is correctly rejected by HEAD. git bisect tells me that this was fixed by Improve type checking of generic function bodies #1580, though I didn't bother to work out why exactly. Thanks for the example, I'll add this as a regression test.

• What it sounds like you're asking for here, in a simpler example, is: Suppose I write a function like

  def search(xs: List[T], val: T = 0) -> bool: ...

  Currently mypy rejects this definition because 0 may not have type T. You're proposing that instead, mypy accept this definition, but then only accept those calls search(xs) for which the default value would be correctly typed (in this case, where T is a superclass of int). When the function is called with an explicit val argument then the default value is irrelevant, so then T is allowed to take on any type.

  This isn't unreasonable but it does make things a little more complicated: we'd need to record the actual types of the default argument expressions (as well as the formal types of the corresponding arguments) in the type of search so that they could be checked for compatibility at call sites. I'm not really sure how I feel about this versus the current behavior. In the case in question here, I think it might be clearer to write an overloaded type anyways (once that is supported).