Add a mechanism to capture diagnostics during code evaluation

[josevalim]

Note that we should also catch diagnostics from the verification pass and normalize warnings coming from IO.warn/2.

[zacck]

@josevalim @whatyouhide would either of you mind breaking down how we could go about adding this mechanism? Perhaps a list of tasks that need to be achieved to accomplish this?

Thank you!

[josevalim]

Sorry but it is not fully clear yet but the goal is to get warnings from Code.eval_quoted commands. I wrote it mostly as a reminder, I will break it down once it is clear in my head.

[zacck]

Hmm maybe I can ask, how does one collect said errors and once one collects the errors how would you like them reported?

…

Sent from my iPhone

On 9 Dec 2022, at 10:20, José Valim ***@***.***> wrote:  Sorry but it is not fully clear yet but the goal is to get warnings from Code.eval_quoted commands. I wrote it mostly as a reminder, I will break it down once it is clear in my head. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[josevalim]

This should also work for code compilation and compile_ commands.

[josevalim]

We should also convert the warning/error data structures into maps.

[scottming]

iex(9)> quoted_ast =
...(9)>   quote do
...(9)>     defmodule MyModule do
...(9)>       f
...(9)>     end
...(9)>   end
{:defmodule, [context: Elixir, imports: [{2, Kernel}]],
 [{:__aliases__, [alias: false], [:MyModule]}, [do: {:f, [], Elixir}]]}
iex(10)> Code.with_diagnostics([log: true], fn -> Code.compile_quoted(quoted_ast, "/unknown.ex") end)
warning: redefining module MyModule (current version defined in memory)
  /unknown.ex: MyModule (module)

error: undefined variable "f" (context Elixir)
  /unknown.ex: MyModule (module)

** (CompileError) /unknown.ex: cannot compile module MyModule (errors have been logged)
    (elixir 1.15.1) src/elixir_expand.erl:383: :elixir_expand.expand/3
    (elixir 1.15.1) src/elixir_expand.erl:10: :elixir_expand.expand/3
    (elixir 1.15.1) src/elixir_expand.erl:538: :elixir_expand.expand_block/5
    (elixir 1.15.1) src/elixir_expand.erl:46: :elixir_expand.expand/3
    (elixir 1.15.1) src/elixir_compiler.erl:44: :elixir_compiler.compile/3
    (elixir 1.15.1) src/elixir_module.erl:408: :elixir_module.eval_form/7
    iex:10: (file)
iex(10)>

Does it not work for Code.compile_quoted? @josevalim

Before version 1.15, we were extracting columns and specific error messages like this:

quoted_ast = 
  quote do
    defmodule MyModule do
      f
    end
  end
try do
  modules = for {m, _b} <- Code.compile_quoted(quoted_ast, "/unknown.ex"), do: m
  {:ok, modules}
rescue
  exception ->
    {filled_exception, stack} = Exception.blame(:error, exception, __STACKTRACE__)
    {:exception, filled_exception, stack, quoted_ast}
end

and this is the result:

{:exception,
 %CompileError{
   file: "/unknown.ex",
   line: 0,
   description: "undefined function f/0 (there is no such import)"
 },
 [
   {:elixir_expand, :expand, 3, [file: 'src/elixir_expand.erl', line: 10]},
   {:elixir_expand, :expand_block, 5,
    [file: 'src/elixir_expand.erl', line: 536]},
   {:elixir_expand, :expand, 3, [file: 'src/elixir_expand.erl', line: 40]},
   {:elixir_compiler, :compile, 3, [file: 'src/elixir_compiler.erl', line: 44]},
   {:elixir_module, :eval_form, 7, [file: 'src/elixir_module.erl', line: 379]},
   {:elixir_module, :compile, 6, [file: 'src/elixir_module.erl', line: 105]},
   {:elixir_eval, :__FILE__, 1, [file: 'iex', line: 17]}
 ],
 {:defmodule, [context: Elixir, imports: [{2, Kernel}]],
  [{:__aliases__, [alias: false], [:MyModule]}, [do: {:f, [], Elixir}]]}}

But in 1.15, we may only be able to use capture_io to get the message information, so it would be nice not to need capture_io or rescue exceptions.

[sabiwara]

@scottming with_diagnostics/2 does not catch errors, but by adding try/rescue you can get the errors:

Code.with_diagnostics([log: true], fn -> 
  try do
    Code.compile_quoted(quoted_ast, "/unknown.ex")
  rescue
    err -> err
  end
end)

gives

{%CompileError{
   file: "/unknown.ex",
   line: 0,
   description: "cannot compile module A (errors have been logged)"
 },
 [
   %{
     file: "/unknown.ex",
     message: "undefined variable \"f\" (context Elixir)",
     position: 0,
     severity: :error,
     stacktrace: [{A, :__MODULE__, 0, [file: "/unknown.ex", line: 0]}]
   }
 ]}

Will add a clarification to the documentation.

[scottming]

@sabiwara Thanks for the quick reply, this could work, but I have one more question, can I only pass a real file to the second parameter if I want to get the exact number of lines?

like:

for file:

defmodule Dummy do
  def hello do
    # :world
    f
  end
end

I should

Code.with_diagnostics([log: true], fn -> 
  try do
    "lib/dummy.ex" |> File.read!() |> Code.string_to_quoted!() |> Code.compile_quoted("lib/dummy.ex")
  rescue
    err -> err
  end
end)

the result will give me position: 4

This would be unfriendly for testing, as we are implementing the new LS. So many tests would require specific columns and rows. I don't want to have a specific file dependency for each test.

[sabiwara]

This would be unfriendly for testing, as we are implementing the new LS. So many tests would require specific columns and rows. I don't want to have a specific file dependency for each test.

Sorry @scottming I don't think I understood your question. AFAIK the file path is just used for reporting (cf docs) but it doesn't have to be a real file and doesn't change the diagnostic lines?

PS: you should be able to call directly Code.compile_string/2 or Code.compile_file/2 instead of the long pipeline 🙂