Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type enclosing query #1304

Merged
merged 9 commits into from
Jul 3, 2024
Merged

Type enclosing query #1304

merged 9 commits into from
Jul 3, 2024

Conversation

xvw
Copy link
Collaborator

@xvw xvw commented Jun 10, 2024

This PR is based on 1265 and add a type enclosing custom request.

@voodoos
Copy link
Collaborator

voodoos commented Jun 19, 2024

I don't remember: is there any reason why this is based on the 5.2 branch instead of master ?

@xvw
Copy link
Collaborator Author

xvw commented Jun 19, 2024

I don't remember: is there any reason why this is based on the 5.2 branch instead of master ?

I am currently rebased on master!

Branch rebased on master.

@xvw xvw changed the title wip: Type enclosing query Type enclosing query Jun 20, 2024
@coveralls
Copy link

coveralls commented Jun 20, 2024

Pull Request Test Coverage Report for Build 4367

Details

  • 63 of 89 (70.79%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.3%) to 22.313%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 63 89 70.79%
Totals Coverage Status
Change from base Build 4363: 0.3%
Covered Lines: 5515
Relevant Lines: 24717

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jun 20, 2024

Pull Request Test Coverage Report for Build 4368

Details

  • 64 of 90 (71.11%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.3%) to 22.313%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 64 90 71.11%
Totals Coverage Status
Change from base Build 4363: 0.3%
Covered Lines: 5515
Relevant Lines: 24717

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jun 20, 2024

Pull Request Test Coverage Report for Build 4369

Details

  • 63 of 89 (70.79%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.273%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 63 89 70.79%
Totals Coverage Status
Change from base Build 4363: 0.2%
Covered Lines: 5505
Relevant Lines: 24716

💛 - Coveralls

@xvw xvw changed the title Type enclosing query wip: Type enclosing query Jun 20, 2024
@coveralls
Copy link

coveralls commented Jun 20, 2024

Pull Request Test Coverage Report for Build 4370

Details

  • 44 of 57 (77.19%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.221%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 44 57 77.19%
Totals Coverage Status
Change from base Build 4363: 0.2%
Covered Lines: 5485
Relevant Lines: 24684

💛 - Coveralls

@xvw xvw force-pushed the type-enclosing-query branch 3 times, most recently from f6e6a53 to 425f6e0 Compare June 20, 2024 14:21
@xvw xvw changed the title wip: Type enclosing query Type enclosing query Jun 20, 2024
@xvw xvw requested a review from voodoos June 20, 2024 14:22
@coveralls
Copy link

coveralls commented Jun 20, 2024

Pull Request Test Coverage Report for Build 4371

Details

  • 45 of 57 (78.95%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.225%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 45 57 78.95%
Totals Coverage Status
Change from base Build 4363: 0.2%
Covered Lines: 5486
Relevant Lines: 24684

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jun 20, 2024

Pull Request Test Coverage Report for Build 4372

Details

  • 47 of 59 (79.66%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.235%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 47 59 79.66%
Totals Coverage Status
Change from base Build 4363: 0.2%
Covered Lines: 5489
Relevant Lines: 24686

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jun 20, 2024

Pull Request Test Coverage Report for Build 4373

Details

  • 47 of 59 (79.66%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.235%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 47 59 79.66%
Totals Coverage Status
Change from base Build 4363: 0.2%
Covered Lines: 5489
Relevant Lines: 24686

💛 - Coveralls

@xvw xvw changed the title Type enclosing query wip: Type enclosing query Jun 24, 2024
@coveralls
Copy link

coveralls commented Jun 24, 2024

Pull Request Test Coverage Report for Build 4380

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 51 of 64 (79.69%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.247%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 51 64 79.69%
Totals Coverage Status
Change from base Build 4363: 0.2%
Covered Lines: 5493
Relevant Lines: 24691

💛 - Coveralls

ocaml-lsp-server/docs/ocamllsp/typeEnclosing-spec.md Outdated Show resolved Hide resolved
| Some position ->
let lend = range.Range.end_ in
if lend.line = position.Position.line then
lend.character > position.character
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
lend.character > position.character
lend.character >= position.character

I think ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really necessary because it is already guarded by the first condition:

let overlap_with_range_end range = function
  | None -> true
  | Some position ->
    let lend = range.Range.end_ in
    if lend.line = position.Position.line (* this one *) then
      lend.character > position.character
    else lend.line > position.line

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure about that ? I don't see it... they could be on the same line and the same character ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh you're right. I was think that your where speaking about line!

}
],
"type": "sig type t = A of int val x : int end"
} |}]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also put a focus on the range_end cases that rely on tricky position comparisons and index shifting :-)

@xvw xvw changed the title wip: Type enclosing query Type enclosing query Jun 24, 2024
@coveralls
Copy link

coveralls commented Jun 24, 2024

Pull Request Test Coverage Report for Build 4382

Details

  • 51 of 64 (79.69%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.247%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 51 64 79.69%
Totals Coverage Status
Change from base Build 4378: 0.2%
Covered Lines: 5493
Relevant Lines: 24691

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jun 25, 2024

Pull Request Test Coverage Report for Build 4383

Details

  • 57 of 70 (81.43%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.266%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 57 70 81.43%
Totals Coverage Status
Change from base Build 4378: 0.2%
Covered Lines: 5499
Relevant Lines: 24697

💛 - Coveralls

Copy link
Collaborator

@voodoos voodoos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's starting to look good !
There is still some weirdness in the tests however...

(Also, but that's personal taste: I think utility functions are sometimes a bit too far from their call point. Especially when they are only used once, sometime you could just inline them or move them in a deeper scope. But that's really nitpicking, don't feel obliged to change anything.)

"position": Position,
"index": uinteger,
"verbosity?": uinteger,
"rangeEnd?": Position
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should describe the roles of the parameters here, especially rangeend (we could simply reuse the comments in the design doc if they are up-to-date)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this scheme, I'd recommend the following:

  1. Either take a range parameter
  2. Or a position parameter

That way you can stick to standard field names and users can use the convenient Range.t type to work with your request.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. Fixed in 89efc55

Comment on lines 115 to 117
let get_logical_position tdp =
let p = tdp.Lsp.Types.TextDocumentPositionParams.position in
Position.logical p
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: not sure about the need for this function. It is far from it's unique callsite and does nothing more than calling a function that already bear a quite similar name.

| Some position ->
let lend = range.Range.end_ in
if lend.line = position.Position.line then
lend.character > position.character
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure about that ? I don't see it... they could be on the same line and the same character ?

Comment on lines 78 to 81
let overlap_with_range_end range position =
let lend = range.Range.end_ in
if lend.line = position.Position.line then lend.character > position.character
else lend.line > position.line
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about that, but maybe we could simply use Position.compare_inclusion to check that the range_end pos is inside the enclosing range. This has the main advantage of not duplicating this kind of error-prone reasoning...

let source = {|let x = string_of_int 2002|} in
let request client =
let open Fiber.O in
let position = Position.create ~line:0 ~character:22 in
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to read the tests: these are positions 0-based right ? For the line it looks like it is the case, and for the column (character) number too ?

Comment on lines 42 to 48
"enclosings": [
{
"end": { "character": 26, "line": 0 },
"start": { "character": 8, "line": 0 }
}
],
"type": "int"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong isn't it ?

Looks like the enclosing of string_of_int 2002 which should have type string...
We expect something like this:

      "index": 0,
      "enclosings": [
        {
          "end": { "character": 26, "line": 0 },
          "start": { "character": 22, "line": 0 }
        },
        {
          "end": { "character": 26, "line": 0 },
          "start": { "character": 8, "line": 0 }
        }
      ],
      "type": "int"
}

And if the required index was 1:

      "index": 1,
      "enclosings": [
        {
          "end": { "character": 26, "line": 0 },
          "start": { "character": 22, "line": 0 }
        },
        {
          "end": { "character": 26, "line": 0 },
          "start": { "character": 8, "line": 0 }
        }
      ],
      "type": "string"
}

| Ordering.(Gt, Eq)
| Ordering.(Eq, Eq)
| Ordering.(Gt, Lt) -> Some i
| _ -> aux (succ i) xs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you expand this pattern?

Query_protocol.Type_enclosing (None, position, Some index)

let get_first_enclosing_index range_end enclosings =
let rec aux i = function
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is just List.findi?

let print_type_enclosing result =
result |> Yojson.Safe.pretty_to_string ~std:false |> print_endline

let test ?(verbosity = 0) ?(index = 0) ?range_end ~line ~character source =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say just make these required

in
Client.request client req

let print_type_enclosing result =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can move this to e2e-new/test.ml since its general purpose

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that really general purpose? FMPOV it is really related to testing type enclosing queries.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears in e2e-new/completion.ml, e2e-new/hover_extended.ml, e2e-new/merlin_call_compatible.ml. So I'd say it's general purpose. You gave it a very specific name here, but it could just be called pretty_print_json

let text_document = TextDocumentIdentifier.create ~uri in
let params =
`Assoc
([ ("textDocument", TextDocumentIdentifier.yojson_of_t text_document)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about just exporting the type from req_type_enclosing? Writing out the json here will just make it harder to sync the test with the actual code.

let dispatch_type_enclosing text_document_position index range_end pipeline =
let position =
Position.logical
text_document_position.Lsp.Types.TextDocumentPositionParams.position
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can annotation text_document_position instead.

| Some (typ, enclosings) ->
(typ, List.map ~f:Lsp.Types.Range.yojson_of_t enclosings)
in
`Assoc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a defined response type.

Some { index; range_end; verbosity; text_document_position }
| _ -> None

let of_yojson_exn = function
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this function decodes params:

  1. Its name should be params_of_yojson
  2. Its signature should be Yojson.t -> params

We must match the conventions of the code generator

@rgrinberg
Copy link
Member

Sorry for the long winded review. Regarding all the issues with json, I realize that these are issues that are essentially present in all of our custom requests. Indeed, we've added a lot of custom code doing things that are standard and all of our standard requests have a quirky behavior. I see that at least somebody tried to unify things a little bit by introducing Custom_request. We should expand that example. To briefly summarize the rules for custom requests:

  1. The overarching rule is that hand written encoders/decoders should be interchangeable with generated ones.
  2. This means that we need to follow all the conventions of generated encoders/decoders
  3. It's good to have explicit request/response types for every single custom request (They're going to be very useful once we need to version things)
  4. The conversion from json decoding errors to the appropriate jsonrpc error should be done once and for all somewhere in jsonrpc-fiber or lsp-fiber. E.g. we handle this for all regular lsp requests in one swoop:
      match In_request.of_jsonrpc req with
      | Error message ->
        let code = Jsonrpc.Response.Error.Code.InvalidParams in
        let error = Jsonrpc.Response.Error.make ~code ~message () in
        Fiber.return
          (Jsonrpc_fiber.Reply.now (Jsonrpc.Response.error req.id error), state)

We should strive for the same for custom requests.

In any case, it's not fair to hold your code to a higher standard to all previous code. So for now you don't need to address to make everything uniform, but I would like at least point 3 addressed with request/response module and their own t_of_yojson and yojson_of_t functions (of whatever type).

@xvw
Copy link
Collaborator Author

xvw commented Jun 29, 2024

@rgrinberg

Thank you very much for your detailed review!

In any case, it's not fair to hold your code to a higher standard to all previous code.

From my point of view, it's not worth merging code that we explicitly know can be improved, so I'm going to try, as best I can, to take into account all the proposed points. Then I'll try to do a pass on all the custom requests that have already been implemented.

Thank you very much, I'll take the liberty of pinging you when this PR is ready (Monday).

@rgrinberg
Copy link
Member

Sure, I'm certainly against improving things across the board. Feel free to regularize all custom requests. I'll give some background why the situation is like this in the first place. In the beginning, I was very much against custom requests and even more so against investing a lot of work into them. I thought they would require a lot duplicate work on the client side and the LSP protocol would evolve to make them all obsolete anyhow. The LSP protocol didn't really evolve much and the use cases for custom requests steadily grew. The end result is that we ended up with a lot of ad-hoc extensions.

@coveralls
Copy link

coveralls commented Jul 1, 2024

Pull Request Test Coverage Report for Build 4395

Details

  • 57 of 79 (72.15%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.254%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 57 79 72.15%
Totals Coverage Status
Change from base Build 4387: 0.2%
Covered Lines: 5498
Relevant Lines: 24706

💛 - Coveralls

@coveralls
Copy link

coveralls commented Jul 1, 2024

Pull Request Test Coverage Report for Build 4397

Details

  • 58 of 79 (73.42%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 22.258%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ocaml-lsp-server/src/custom_requests/req_type_enclosing.ml 57 78 73.08%
Totals Coverage Status
Change from base Build 4387: 0.2%
Covered Lines: 5499
Relevant Lines: 24706

💛 - Coveralls


let%expect_test {|
First, let's locate on [A.z], we expect the type [t], but we
will increase the verbosity in order to get the fuill expansion of
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
will increase the verbosity in order to get the fuill expansion of
will increase the verbosity in order to get the full expansion of

Copy link
Collaborator

@voodoos voodoos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is ready :-)

We should open an issue in merlin about the stability of enclosings when index is 0 or 1.

ocaml-lsp-server/docs/ocamllsp/typeEnclosing-spec.md Outdated Show resolved Hide resolved
@voodoos
Copy link
Collaborator

voodoos commented Jul 3, 2024

The CI failures appear to be unrelated ? Maybe we should bump the setup-ocaml action like in ocaml/merlin#1791 ?

Co-authored-by: Ulysse <5031221+voodoos@users.noreply.github.com>
@voodoos voodoos merged commit b73d822 into ocaml:master Jul 3, 2024
5 of 8 checks passed
voodoos added a commit to voodoos/opam-repository that referenced this pull request Jul 5, 2024
CHANGES:

## Features

- Introduce a configuration option to control dune diagnostics. The option is
  called `duneDiganostics` and it may be set to `{ enable: false }` to disable
  diagnostics. (ocaml/ocaml-lsp#1221)

- Support folding of `ifthenelse` expressions (ocaml/ocaml-lsp#1031)

- Improve hover behavior (ocaml/ocaml-lsp#1245)

  Hovers are no longer displaye on useless parsetree nodes such as keywords,
  comments, etc.

  Multiline hovers are now filtered away.

  Display expanded ppx's in the hover window.

- Improve document symbols (ocaml/ocaml-lsp#1247)

  Use the parse tree instead of the typed tree. This means that document
  symbols will work even if the source code doesn't type check.

  Include symbols at arbitrary depth.

  Differentiate functions / types / variants / etc.

  This now includes PPXs like `let%expect_test` or `let%bench` in the outline.

- Introduce a `destruct-line` code action. This is an improved version of the
  old `destruct` code action. (ocaml/ocaml-lsp#1283)

- Improve signature inference to only include types for elements that were
  absent from the signature. Previously, all signature items would always be
  inserted. (ocaml/ocaml-lsp#1289)

- Add an `update-signature` code action to update the types of elements that
  were already present in the signature (ocaml/ocaml-lsp#1289)

- Add custom
  [`ocamllsp/merlinCallCompatible`](https://github.com/ocaml/ocaml-lsp/blob/e165f6a3962c356adc7364b9ca71788e93489dd0/ocaml-lsp-server/docs/ocamllsp/merlinCallCompatible-spec.md)
  request (ocaml/ocaml-lsp#1265)

- Add custom [`ocamllsp/typeEnclosing`](https://github.com/ocaml/ocaml-lsp/blob/109801e56f2060caf4487427bede28b824f4f1fe/ocaml-lsp-server/docs/ocamllsp/typeEnclosing-spec.md) request (ocaml/ocaml-lsp#1304)

## Fixes

- Detect document kind by looking at merlin's `suffixes` config.

  This enables more lsp features for non-.ml/.mli files. Though it still
  depends on merlin's support. (ocaml/ocaml-lsp#1237)

- Correctly accept the `--clientProcessId` flag. (ocaml/ocaml-lsp#1242)

- Disable automatic completion and signature help inside comments (ocaml/ocaml-lsp#1246)

- Includes a new optional/configurable option to toggle syntax documentation. If
  toggled on, allows display of syntax documentation on hover tooltips. Can be
  controlled via environment variables and by GUI for VS code. (ocaml/ocaml-lsp#1218)

- For completions on labels that the LSP gets from merlin, take into account
  whether the prefix being completed starts with `~` or `?`. Change the label
  completions that start with `?` to start with `~` when the prefix being
  completed starts with `~`. (ocaml/ocaml-lsp#1277)

- Fix document syncing (ocaml/ocaml-lsp#1278, ocaml/ocaml-lsp#1280, fixes ocaml/ocaml-lsp#1207)

- Stop generating inlay hints on generated code (ocaml/ocaml-lsp#1290)

- Fix parenthesizing of function types in `SignatureHelp` (ocaml/ocaml-lsp#1296)

- Fix syntax documentation rendering (ocaml/ocaml-lsp#1318)
@xvw xvw deleted the type-enclosing-query branch July 8, 2024 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants