Add explanation for errors caused by a type constraint propagated from a keyword #1510

Armael · 2017-12-03T12:11:39Z

Another change extracted from PR #102 . This adds specific explanation for type errors that involve type information propagated by a keyword such as if, for, while...

For example,

if true then 42

highlights "42" and prints:

Error: This expression has type int but an expression was expected of type
         unit because it is the result of a conditional with no else branch

The idea for this patch comes from #102 but the implementation proposed here is different. The original patch changed the order of propagation in order to get more precise information -- but this doesn't interact well with non-principality AFAIU. The approach followed in this patch is as follows: in the type_expect function, we add an "explanation" optional argument. If unification fails between something and the expected type ty_expected, then this explanation (if present) explains why this type was expected (e.g. "because it was the body of a for-loop"). When type_expect calls itself recursively with a different expected type, the explanation is dropped.

gasche · 2017-12-03T12:30:46Z

testsuite/tests/typing-core-bugs/type_expected_explanation.ml

+         bool because it is the condition of a if-statement
+|}];;
+
+fun b -> if true then (if b then ()) else (print_int b);;


You should explicit point out that this is a case where the refined message doesn't show up because of propagation order.

gasche · 2017-12-03T12:50:29Z

typing/typecore.ml

  let loc = sexp.pexp_loc in
  (* Record the expression type before unifying it with the expected type *)
+  let unify_with_explanation = unify_with_explanation explanation in


This should be with_explanation

gasche · 2017-12-03T12:51:40Z

typing/typecore.ml

+
+and type_expect_
+    ?in_function ?(recarg=Rejected) ?explanation
+    env sexp ty_expected =


Instead of a separate explanation argument, have you considered carrying the information within ty_expected, turning it into a pair of a type and an optional explanation?

gasche · 2017-12-03T13:53:40Z

Some more comments on the design choices:

The explanation type is actually a list of approximations of program contexts, that explain why we propagate a certain expected type. For example, you propagate bool when you go under the context if [_] then e1 else e2, where [_] denotes the hole of the sub-term that is going to be type-checked.

Arthur's implementation is careful to delay the unification of the expected type until the last possible moment. This implementation does not do it, which means that in some case the explanation will not be used while it should be. If you write if b then e1 else e2 and b was already determined to have type int, there will be an error when unifying b's type with bool, and the explanation will be given. But if b has an unknown point at this time, the unification will succeed and the unification will be dropped, while a different propagation order could have resulted in the error being detected there (and will be reliably detected there with Arthur's change-of-propagation-order approach).
One example is the term fun b -> if true then (if b then 1 else 2) else (print_int b): one of the two branches of if true will be type-checked first, and the refined explanation will or will not be used depending on the order (so if you also look at the term with inversed branch order, then one of them will have a nice error message and the other won't.)

The reasoning for having this feature even though the propagation order makes it more fragile than with Arthur's approach is:

It is already helpful in the cases where it works (and it works in a lot of situations that beginners may encounter, see the tests).
if we later decide to change the type propagation order along Arthur's line (and find a way to make it not break most programs), the work by Armaël in this PR can be reused as-is in this new world, it won't need big changes. So there is no clear reason to delay the merge of this part.

Finally: Armaël commented above that the explanation is not reused for recursive calls to the type-checker to a different expected type. Of course! The context of those sub-calls is different so it would be incorrect to reuse the explanation. The question is rather of why it is correct to reuse the explanation for the sub-calls with the same expected types. For example, if you have a program of the form if (a; b) then e1 else e2, then (a; b) will be type-checked with the if [_] then e1 else e2 explanation, and in fact b will be type-checked recursively with the same explanation, while technically its own context is rather if (a; [_]) then e1 else e2.

In the case of moving from if [_] to if (a; [_]), we considered that the error message for the if context was vague enough that this would not be confused. If b is highlighted with the explanation "this is the condition of an if", users will understand, even if technically (a; b) is the condition. Armaël and I reviewed each case of recursive sub-call with same expected type, and we considered that, each time, the approximation is reasonable. We could define a more refined type of type-forcing contexts (so the explanation would be eg. Sequence_right (If_condition) instead of just (If_condition), and print more precise messages, but we judged that it was not worth it.

This is obvious for if- and while-conditions of for-bounds for examples, which are always very small expressions anyway. It is less clear for the context while cond do [_] done, which forces unit. For example if you write

while cond do
  match foo with
  | K1 -> ()
  | K2 -> 2
  | K3 -> ()
done

then 2 will have the error explanation "... expected of type unit because it is the body of a while". (TODO @Armael: use "it is in the body of a while loop" instead.)

alainfrisch · 2017-12-04T13:24:53Z

typing/typecore.mli

@@ -49,7 +66,7 @@ val check_partial:
        Location.t -> Typedtree.case list -> Typedtree.partial
 val type_expect:
        ?in_function:(Location.t * type_expr) ->
-        Env.t -> Parsetree.expression -> type_expr -> Typedtree.expression
+        Env.t -> Parsetree.expression -> type_expected -> Typedtree.expression


Wouldn't it be lighter to pass the explanation as an (optional) extra argument, instead of wrapping it in a record with the expected type?

Well, that's what I was doing at first, but then @gasche suggested that I attach the explanation to the expected type...

More precisely, commit 12d26f3 implements @gasche suggestion, so the commits before that correspond to what you mention.

Ah ok, sorry, I did not see @gasche 's comment. I've a preference for the old version, but I don't mind if the new one is preferred. If we go for the record solution: (i) would it make sense to include in_function in it? (ii) perhaps use a function to build the record instead of using literal; it will simplify adding more optional fields later if needed.

My intuition is that we will more towards more and more structured representations of the bidirectional information propagated during type-checking, and that having it packed in one value is the right way for nice code evolution.

I just pushed a commit that provides a function to build the record as @alainfrisch suggested.

sliquister · 2017-12-04T14:19:18Z

I wonder if, in general, these messages talking about syntax will end up being undesirable in generated code.

I suppose they can be disabled by replacing [if e1 then e2] by [if e1 then e2 else ()], or [while b do e done] by [while (b : bool) do e; (); done] etc, so it's not too bad.

But with the rec one, the hints are going to be weird (not tested with an actual ppx, but the result should be the same), since it will say "you are missing the rec keyword" on a line that says type nonrec:

type sexp

type t (* forgot [@@deriving sexp_of] *)

module M = struct
  type nonrec t = t [@@deriving sexp_of]
  let sexp_of_t : sexp -> t = sexp_of_t
end

Error: Unbound value sexp_of_t
      Hint: You are probably missing the `rec' keyword on line 7.

Armael · 2017-12-04T14:51:29Z

One way of addressing that issue would be to add a let nonrec construction; which would be used by ppx_deriving when called on a type nonrec. Then, the "missing rec" error message could be disabled on lec nonrec constructions.
I assume pull requests implementing this are welcome :-).

sliquister · 2017-12-05T04:59:24Z

It's clearly not worth adding syntax for my concern. I suppose the rec hint could be disabled as well if necessary by adding some let _ = sexp_of_t before the real code.

Armael · 2017-12-05T15:05:10Z

I had forgotten to add an explanation for the left-hand side of a sequence. The last commit implements this.

Armael · 2017-12-05T17:06:34Z

I tweaked the formatting to insert a break hint before the "because it is ..." explanation. This is probably better that no break hint; however now the messages in the tests look a bit weird, because of the staggering produced by the structural hovbox that contains the messages.

gasche · 2017-12-06T07:34:57Z

typing/typecore.ml

@@ -3163,7 +3164,8 @@ and type_expect_
            exp_env = env }
      end
  | Pexp_sequence(sexp1, sexp2) ->
-      let exp1 = type_statement env sexp1 in
+      let exp1 = type_statement ~explanation:Sequence_left_hand_side
+          env sexp1 in


I would have expected a test for -strict-sequence to show up in the patch. Where does this happen?

It doesn't appear in this patch because it is already present in type_statement which was modified previously (loop bodies are handled in exactly the same way as the lhs of a sequence).

Drup · 2017-12-07T11:49:18Z

Isn't it one of those cases where we should check if the associated expression is ghost before using it as an explanation ?

gasche · 2017-12-08T11:55:10Z

@Drup: we agree with that proposal, thanks! @Armael plans to implement it.

@alainfrisch would you be willing to "drive" this issue, and in particular approve it if you do approve of the proposed change? (I have handled previous change proposals by Armaël but I'm basically a co-author of this one so I don't think it would be best.)

alainfrisch · 2017-12-08T13:15:17Z

would you be willing to "drive" this issue

I only looked superficially to the implementation, and don't have any opinion on the proposal itself. I think @gasche would do a better job here (as often, and unfortunately for him).

Armael · 2017-12-10T16:55:03Z

I added a commit that (I think) implements @Drup suggestion.

gasche · 2017-12-10T18:52:42Z

(I believe this commit is correct as well.)

The search for a voluntary review goes on: @sliquister, @Drup, would one of you be willing to review this patch and formally "approve" it if you believe it is correct?

gasche · 2017-12-12T14:21:04Z

( @let-def, maybe you would be interested in reviewing this as well? )

Octachron · 2017-12-12T19:54:45Z

I am wondering if ? should count as a keyword in:

let f g = g ?x:1 ();;

Error: This expression has type int but an expression was expected of type
'a option

let-def

The implementation looks good to me.
I don't have a strong opinion about the effect on end-users, though I believe it is a improvement.

let-def · 2017-12-13T04:15:42Z

typing/typecore.mli

+type type_expected = private {
+  ty: type_expr;
+  explanation: type_forcing_context option;
+}


For future reference, would you mind adding a small comment to explain the point of type_forcing_context and type_expected?
At least that type_forcing_context is not affecting type checking itself but is used for generating error messages.

let-def · 2017-12-13T06:10:28Z

testsuite/tests/typing-core-bugs/type_expected_explanation.ml

+Line _, characters 3-4:
+Error: This expression has type int but an expression was expected of type
+         bool
+       because it is in the condition of a if-statement


an if statement?

Armael · 2017-12-13T14:17:07Z

@let-def : thanks for your comments, I just added commits that address them (I think).

I am wondering if ? should count as a keyword in [optional arguments]

@Octachron : maybe -- though after a quick glance at the existing code, supporting that doesn't seem to be an simple extension of the current patch. So I think it better belongs in a separate PR.

gasche · 2017-12-13T14:18:51Z

There is a minor typo your last commit message: cnotext -> context.

(I wonder if your comments shouldn't go in the .mli rather the .ml. I think that both are fine, but maybe you could manage to start a trend of actually, you know, documenting interfaces, and that would be Big!)

Armael · 2017-12-13T14:21:43Z

Good point. I amended the last commit and put the comments in the .mli rather than in the .ml.

Octachron · 2017-12-13T20:36:29Z

maybe -- though after a quick glance at the existing code, supporting that doesn't seem to be an simple extension of the current patch. So I think it better belongs in a separate PR.

Thanks for having a look, I was just asking in case it was an one-liner extension.

Armael · 2017-12-14T07:23:15Z

I rebased on top of trunk, to benefit from the super awesome error highlighting in expect tests.

Armael · 2017-12-21T10:20:46Z

Thanks for the approval @let-def ! Should I rebase on top of trunk and clean-up the history now?

gasche · 2017-12-21T10:48:18Z

Yes. (I didn't ping explicitly as I didn't know how much OCaml you wanted during your holidays.)

Armael · 2017-12-21T10:58:50Z

Rebased.

Armael force-pushed the type_expect_unif_explanation branch from cf803a1 to 3b3ddc7 Compare December 3, 2017 12:16

gasche reviewed Dec 3, 2017

View reviewed changes

Armael force-pushed the type_expect_unif_explanation branch 5 times, most recently from e0ea009 to 597bc79 Compare December 3, 2017 16:02

alainfrisch reviewed Dec 4, 2017

View reviewed changes

gasche reviewed Dec 6, 2017

View reviewed changes

gasche mentioned this pull request Dec 11, 2017

This expression has type <x> but an expression was expected of type unit reasonml-community/error-message-improvement#30

Closed

let-def reviewed Dec 13, 2017

View reviewed changes

Armael force-pushed the type_expect_unif_explanation branch from c96053d to b2e7c75 Compare December 13, 2017 14:21

Armael force-pushed the type_expect_unif_explanation branch from b2e7c75 to 73d4e8b Compare December 14, 2017 07:21

let-def approved these changes Dec 21, 2017

View reviewed changes

Armael added 3 commits December 21, 2017 11:50

Add specific error messages in case of type propagated from a keyword

ea00ce0

Add tests

642d5c6

Update Changes

55624d7

Armael force-pushed the type_expect_unif_explanation branch from 73d4e8b to 55624d7 Compare December 21, 2017 10:58

gasche merged commit c8c4fb5 into ocaml:trunk Dec 21, 2017

gasche mentioned this pull request Mar 24, 2018

Improved error messages #102

Open

Armael mentioned this pull request Jul 4, 2018

Add an explanation for the expected type of when-clauses #1884

Merged

Armael mentioned this pull request Aug 6, 2018

Enable -strict-sequence by default #1971

Open

Add explanation for errors caused by a type constraint propagated from a keyword #1510

Add explanation for errors caused by a type constraint propagated from a keyword #1510

Armael commented Dec 3, 2017 •

edited

gasche Dec 3, 2017

gasche Dec 3, 2017

gasche Dec 3, 2017

gasche commented Dec 3, 2017

alainfrisch Dec 4, 2017

Armael Dec 4, 2017

Armael Dec 4, 2017

alainfrisch Dec 4, 2017

gasche Dec 4, 2017

Armael Dec 5, 2017

sliquister commented Dec 4, 2017

Armael commented Dec 4, 2017

sliquister commented Dec 5, 2017

Armael commented Dec 5, 2017 •

edited

Armael commented Dec 5, 2017

gasche Dec 6, 2017

trefis Dec 6, 2017

gasche Dec 6, 2017

Drup commented Dec 7, 2017

gasche commented Dec 8, 2017

alainfrisch commented Dec 8, 2017

Armael commented Dec 10, 2017

gasche commented Dec 10, 2017

gasche commented Dec 12, 2017

Octachron commented Dec 12, 2017

let-def left a comment

let-def Dec 13, 2017

let-def Dec 13, 2017

Armael commented Dec 13, 2017 •

edited

gasche commented Dec 13, 2017

Armael commented Dec 13, 2017

Octachron commented Dec 13, 2017

Armael commented Dec 14, 2017

Armael commented Dec 21, 2017

gasche commented Dec 21, 2017

Armael commented Dec 21, 2017

Add explanation for errors caused by a type constraint propagated from a keyword #1510

Add explanation for errors caused by a type constraint propagated from a keyword #1510

Conversation

Armael commented Dec 3, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gasche commented Dec 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sliquister commented Dec 4, 2017

Armael commented Dec 4, 2017

sliquister commented Dec 5, 2017

Armael commented Dec 5, 2017 • edited

Armael commented Dec 5, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Drup commented Dec 7, 2017

gasche commented Dec 8, 2017

alainfrisch commented Dec 8, 2017

Armael commented Dec 10, 2017

gasche commented Dec 10, 2017

gasche commented Dec 12, 2017

Octachron commented Dec 12, 2017

let-def left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Armael commented Dec 13, 2017 • edited

gasche commented Dec 13, 2017

Armael commented Dec 13, 2017

Octachron commented Dec 13, 2017

Armael commented Dec 14, 2017

Armael commented Dec 21, 2017

gasche commented Dec 21, 2017

Armael commented Dec 21, 2017

Armael commented Dec 3, 2017 •

edited

Armael commented Dec 5, 2017 •

edited

Armael commented Dec 13, 2017 •

edited