'infer' in mapped type may lose specificity

[aloifolia]

TypeScript Version: 3.8.3 - 3.9.5 (probably more)

Code

// suppose that IRL this type does something more useful 
type MakeOptional<TObject extends object> = {
  // the "extends infer TValue" is crucial - without it, everything works as expected
  [TKey in keyof TObject]?: TObject[TKey] extends infer TValue
    ? TValue
    : never;
};

type BaseType = { a: number; b: { c: string } };
type ConcreteType = { a: 42; b: { c: 'stuff' }; };

declare const value: ConcreteType;

declare function doNothing<
  // Here, ConcreteType would entail the same result
  TValue extends BaseType, 
  TMappedValue extends MakeOptional<TValue>
>(
  value: TValue,
  mappedValue: TMappedValue,
): {
  value: TValue;
  mappedValue: TMappedValue;
};

const result = doNothing(value, {
  a: 42,
});

Expected behavior:
The inferred type of result should be

{
    value: ConcreteType;
    mappedValue: {
        a: number
    };
}

Actual behavior:
The inferred type of result is

{
    value: ConcreteType;
    mappedValue: MakeOptional<ConcreteType>;
}

There is a workaround: If you pass the argument for the parameter mappedValue with as const, the compiler will instead infer the concrete type. However, if you do this you lose code completion for the fields of mappedValue. So basically, you are left with what I would consider another bug/shortcoming of the compiler.

Playground Link:
https://www.typescriptlang.org/play/index.html?ssl=15&ssc=18&pln=15&pc=26#code/C4TwDgpgBAsghgawgeTMAlgewHZwDYA8AKsgEYBWEAxsFBAB7ATYAmAzlJhdcAHxQBeKAG8AUFCgBtIgGkIIKOmxQkITADMoJbjQC6AfgBcWspRrS5IXXUbN2i7OogAnLQDV8AVwjiJUfe5ePn5QxtgQAG4uANyiAL6xoqCQUADCOFTOEExE4NBCYhJwxgAsAEyxEqTGwlBUxgDkbMCe6uoNUAnxiSzUeHBZdTjNUBFBxunYmdkQuZA9fQPQ6p5TGDhQLJgAcpjAABZKAOYEvkQeeN42TKwck9M5eQA0Z-BgkCwXVww39vBIqHWuEI5yCvFEvAAFL4xpcIMZQXCXhIALZwd4QT7jLRvD5fCAvACUNRh2MR3kqUDRGKxcIRuMx+NiXVEVGGtCybE8eFoQi2uwOx0hsO8TxEvmKUHKLzihOiQA

Related Issues:

[RyanCavanaugh]

I don't understand the argument for why this is a defect. There are two type arguments with a relationship implied by the constraint, and TS satisfied those constraints. In general this sort of cross-parameter inference is the preferred behavior and we've done work to make it work more like people expect.

If you can justify why you would write T extends infer U ? U : never (which seems to be a no-op?) and some kind of implementation bug that would justify this, I can look again, but this is a correct inference as far as TS's rules are concerned.

[aloifolia]

I probably need to clarify what exactly looks broken to me. So, there are two cases:

1. The code as you see it in the playground.
   If you inspect the inferred type of result (by hovering over the variable name), you see this:

2. If you change line 3 from ? TValue to ? TObject[TKey], you see this:

So the field mappedValue in result has a different type, based on whether you used the inferred type or not, although this is basically a no-op (as you confirmed) and should be the same. See this playground where both field types are compared. So the problem is probably not the general behaviour of MakeOptional per se but the type inference when used in the context of the following lines with the doNothing function.

In this basic example this no problem, but I have some code where line 3 is a more complex structure which uses multiple nested infer expressions to keep the code legible.

[aloifolia]

In this playground you see that the field result.mappedValue.a is inferred as being optional although I would expect it to be non-optional.

[RyanCavanaugh]

In this playground you see that the field result.mappedValue.a is inferred as being optional although I would expect it to be non-optional.

I strongly disagree based on this example:

// Error-free implementation of 'doNothing'
function doNothing<TValue extends BaseType, TMappedValue extends MakeOptional<TValue>>(value: TValue,  mappedValue: TMappedValue): {
  value: TValue;
  mappedValue: TMappedValue;
} {
  return { value, mappedValue };
}

// Legal call to 'doNothing'
const result = doNothing<ConcreteType, Partial<ConcreteType>>(value, {});
// Crashes
result.mappedValue.a.toFixed();

[aloifolia]

I think that I still have not quite pointed out which aspect is wrong (in my opinion). So, based on your example, I extended the Playground code.

1. Playground with code that works as expected

Here, the type MakeOptional just returns TObject[TKey]. This leaves us with the following result:

✅ This is good, because the compiler infers that r1.mappedValue.a is non optional whereas r2.mappedValue.a is optional. So both of our example calls work as desired.

2. Playground with code that works in unexpected ways

Here, the type MakeOptional just returns TObject[TKey]. This leaves us with the following result:

❌ This is bad because now both r1.mappedValue.a and r2.mappedValue.a are inferred as optional fields. r1 is unnecessarily imprecise.

The puzzling thing here is that the difference in the inference of the function argument arises from something seemingly unrelated (by returning an infered "copy" of a type vs. just returning the same type in MakeOptional). Also, if you use one of the other "workarounds" that I mentioned in the first post, the compiler will infer the more precise type.

[RyanCavanaugh]

It looks like what's going on there is that the T extends infer U ? U : never is creating a lower-priority inference site, which is a normal part of inference.

[aloifolia]

So, I investigated a bit more to get a feeling for which cases break and which work. In this playground you can see the results. And here are my findings (I will reference the playground code):

1. If you provide both arguments as object literals, the compiler might infer the more specific variant:

• ✅ if the first parameter is typed as an extension of a certain base type, this works → see r1
• ❌ if the first parameter is typed as an extension of a type which is exactly the same type as the provided argument, this does not work → see r7

2. If you use a type cast for the first parameter, the compiler might infer the more specific variant:

• ✅ if you use a cast to the base type (and the function parameter is typed accordingly), this works r2
• ❌ if you use a cast to the concrete type, this does not work → see r3 and r8
• note that in the playground, the typecast examples use literal objects for the first parameter - these can be swapped to variable identifiers without a change of the result

3. If you provide the first argument as a variable identifier, the compiler might the more specific variant:

• ✅ if the variable was initialized with a value, this works → see r5 (here, the fields do not have literal types but more general ones, but this is to be expected - this is documented somewhere)
• ❌ if the variable was assigned a designated type, this does not work initializedValue → see r4

4. ✅ If you provide the second argument with a constant assertion, every one of the examples work → see r6 (this approach can be used for any of the other cases)

After letting all these examples sink in, I come to the following conclusion:

• the current behaviour might follow a certain consistent logic
• for me as a Typescript user, however, this logic is entirely inscrutable and unpredictable

So, with the current state of the compiler, if I want to add a function like doNothing to an API, I have the following choices:

• let user figure out which kind of arguments could possibly infer specific types - basically by trial and error (because they will certainly not understand what's going on)
• tell the user to always use constant assertions if the type inference "breaks" - which means that they may need to struggle with this bug
• let go of uses of the infer keyword in types like MakeOptional- thus, missing the convenience of a generally very useful Typescript feature

The first option is a no-go, the second one would be acceptable (for my case, but still be subpar for the general case) as soon as the linked bug is solved, the third one might work, but might also severly limit the structure and expressiveness of types that I can use. The best option (in my eyes) would be to somehow improve the inference to have consistently specific resulting types.

[RyanCavanaugh]

What you're proposing is basically "TypeScript should just do the right thing during generic inference"

The inference rules started out pretty simple. I could write them down on a small index card.

Then people showed up and said "Hey, inference isn't doing a good thing here, when X > Y and Y > Z then Z < X so therefore you should pick X or Z depending", and we agreed, and made the inference rules a little more complex to make inference do the right thing in that situation.

Then more people showed up and said "Hey, inference isn't doing the right thing here, when there are two inference candidates, this one is clearly superior and should be chosen", and we agreed, and made the inference rules a little more complex to make inference do the right thing in that situation.

Then more people showed up and said "Hey, inference isn't doing the intuitive thing here, when there are two inference candidates and one is from a contravariant inference, then you pick the other candidate instead", and we agreed, and made the inference rules a little more complex to make inference do the right thing in that situation.

This happened dozens more times, based on user feedback, each with the goal of TS just doing the right and intuitive thing.

[aloifolia]

I get your point - the inference mechanism could probably be made arbitrarily complicated and eventually unmaintainable.

So, for me, one of the core aspects of any language is predictability. For any language this means that you should generally be able to see a certain piece of code and tell what it might do. Of course, the more complex the code is or the more side effects it involves the less likely this assumption will materialize.

Now, for a statically typed language with type inference, the surface area includes static types: You should be able to reason about which types are inferred at a certain position in your code (is this assumption naive?) since these types are basic building blocks of your program. If that is not the case, the coding process becomes somewhat of a lottery.

I think that two main factors of type predictability are:

1. Consistency: Do equal expressions entail the same types? Note that I use the somewhat fuzzy "equal" instead of "identical". So the question whether this statement should hold true for a specific example is probably a bit subjective.
2. Traceability: Will a user be able to understand the reasoning of a certain inference? This will probably depend on two factors:
   (a) the experience of the programmer
   (b) the documentation of the programming language

Ideally, every decision that a compiler comes to is completely consistent in the eyes of a slightly experienced programmer. However, this ideal is probably unrealistic for a type system like the one that Typescript establishes. Thus, I suggest that to improve the developer experience nevertheless that the inference rules of Typescript should somehow be documented. This would certainly help library authors avoid constructs which ultimately confuse users. Also, the question of which remaining gaps in the inference should be closed eventually would become more obvious.

[RyanCavanaugh]

We can agree that two identical programs should have identical behavior, and that guarantee is upheld. TypeScript is deterministic even if it's not predictable to a lay user.

However, one programmer's "consistency" is another programmer's "lack of expressiveness" - at the point that two programs diverge in their text, we can (though not necessarily should) assume that the divergence is grounded in some intent to write a different program.

An example is that T extends U ? V : W is different from [T] extends [U] ? V : W due to distributivity rules around bare type parameters. One person could look at that and say "It's inconsistent; TypeScript would be better if it were more consistent and treated these different programs identically". Another person could say "This lets me be expressive by allowing me to control whether my conditional type is distributive or not; it would be worse for TS to remove this feature because it would remove my ability to make this decision myself'.

More documentation is always better but there will always be a point where no reasonable programmer is able to keep all the rules in their head at once and accurately predict the outcomes; the example in the OP is well past what I would expect 99.99% of programmers to be able to reason about. An example with more complexity than a human can manage will always be constructable no matter how well-documented and intuitive the rules are.

[aloifolia]

Exactly, that is what I meant with "consistency is subjective" - which is why I was wondering whether "documentation of inference rules" could help in my example. You might be right that the OP example is already beyond manual human reasonableness - to validate this statement, I would actually need to know the rules ;-).

Spontaneous idea: a tool which, given some code, incrementally shows which inference rules are used and which intermediate types are produced by them. Since I have no clue how the inference mechanism in the compiler is implemented, I don't know if that is even feasible.

To illustrate the last point with the inference helper tool: Based on the OP example, I just assembled this playground:

type MakeOptional<TObject extends object> = {
  [TKey in keyof TObject]?: TObject[TKey] extends infer TValue
    ? TValue
    : never;
};

declare function doNothing<
  TValue extends { a: 42 },
  TMappedValue extends MakeOptional<TValue>
>(value: TValue, mappedValue: TMappedValue): TMappedValue;

const r = doNothing({ a: 42 }, { a: 42 })
const r1 = doNothing({ a: 42 }, { a: 42 } as {a: 42})     // valid cast
const r2 = doNothing({ a: 42 }, { a: 42 } as {a: number}) // invalid cast

r.a.toExponential();  // does not work
r1.a.toExponential(); // works
r2.a.toExponential();

The difference between the calls for r and r1 are not really obvious - what is { a: 42} being inferred to in the call for r? It cannot be the type {a: 42} because then the compiler would not complain about r.a.toExponential() (it would behave like r1). Also, it cannot be inferred as the type that is used for r2 because that would result in a compile time error. So there must be a certain third type that I am unaware of which an inference helper tool could tell me about (I guess).