Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate Cthulhu with VSCode - show types in source code #469

Merged
merged 50 commits into from
Aug 20, 2023

Conversation

Zentrik
Copy link
Collaborator

@Zentrik Zentrik commented Jun 15, 2023

Depends on julia-vscode/julia-vscode#3328 for showing types inline, however can be merged in before.

This pr show types in the source code when using the repl in VSCode, e.g. @descend f(1) with warnings on gives
image
And without warnings
image

There a couple improvements I still want to make to this pr:

  • Adding a way to toggle the vscode integration (should this be implemented the same way the warn toggle works?)
  • Removing the duplication of tests in TypedSyntax (I tried putting the tests into a separate file and using include but then the m = @which TSN.unnamedargs(Matrix{Float32}, Int; a="hello") test started failing claiming mi was nothing)
  • Adding a way to automatically show types in VSCode for any called functions in an open file without having to descend into them.

As an example of how to change the colour of the hints in VSCode, I'm using the Monokai theme with the following in my settings.json

"workbench.colorCustomizations": {
        // Name of the theme you are currently using
        "[Monokai]": {
            // Overrides for specific kinds of inlay hints
            "editorInlayHint.typeForeground": "#66D9EF",
            "editorInlayHint.typeBackground": "#00000000",
            "editorInlayHint.parameterForeground": "#f8f8f2", // defaults from https://github.com/microsoft/vscode/blob/2e335d2df2001439b09483e7fcff8379ac1a7a16/extensions/theme-monokai/themes/monokai-color-theme.json#L41-L42
            "editorInlayHint.parameterBackground": "#75715E",
            // Colours for unstable types when warnings are on
            "editorInlayHint.foreground": "#e13030",
            "editorInlayHint.background": "#00000000",
        }
    }

@timholy
Copy link
Member

timholy commented Jun 15, 2023

I haven't looked at the implementation yet, but I'm very excited to see this! I agree with the items on your TODO list that I understand. What's the precise issue about toggling the integration? Would that be something controlled from the VSCode GUI?

@Zentrik
Copy link
Collaborator Author

Zentrik commented Jun 15, 2023

I don't think there is any good way to toggle the integration from VSCode, the setting for the types and the warning diagnostics (the orange line saying Unstable Type) is Julia wide.
I was thinking the best way would be to add another toggle to @descend like the [w]arn one.

@Zentrik Zentrik marked this pull request as ready for review June 16, 2023 16:59
@codecov-commenter
Copy link

codecov-commenter commented Jun 16, 2023

Codecov Report

Merging #469 (1c21e25) into master (c23d163) will not change coverage.
Report is 4 commits behind head on master.
The diff coverage is 0.00%.

@@          Coverage Diff           @@
##           master    #469   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files           9       9           
  Lines        1433    1512   +79     
======================================
- Misses       1433    1512   +79     
Files Changed Coverage Δ
src/Cthulhu.jl 0.00% <0.00%> (ø)
src/codeview.jl 0.00% <0.00%> (ø)
src/preferences.jl 0.00% <0.00%> (ø)
src/reflection.jl 0.00% <0.00%> (ø)
src/ui.jl 0.00% <0.00%> (ø)

@pfitzseb
Copy link
Member

pfitzseb commented Aug 8, 2023

As Cthulhu already prints a hyperlink to the current function and because it seems like it would be annoying if the editor jumped everytime Cthulhu was used, for now I haven't implemented the second suggestion though I would be happy to if more people wanted it.

I feel like the redundancy of having the whole function printed in the REPL is more annoying than having the editor jump to the function you've descended into -- that workflow matches the normal debugging experience pretty closely. We could maybe add a [F]ollow in editor toggle?

At some point I'd also love to do this "properly" and use the debugger UI for this (additionally printing IR/native code in a separate pane to the side), but that's a bigger project.

Don't revisit already processed callsites and show info when a method definition will not be annotated with VSCode integration.
`get_typed_sourcetext` (specifically the call to  `definition`) is slow and this eliminates quite a few calls to it.
@Zentrik
Copy link
Collaborator Author

Zentrik commented Aug 10, 2023

he redundancy of having the whole function printed in the REPL is more annoying than having the editor jump to the function you've descended into -- that workflow matches the normal debugging experience pretty closely. We could maybe add a [F]ollow in editor toggle?

Is there a way to have the editor jump to a file without switching focus away from the terminal, I presume this is the desired behavior as otherwise it makes Cthulhu a bit annoying to use. I see that https://code.visualstudio.com/api/references/vscode-api#window.showTextDocument allows the terminal to preserve focus, though this doesn't seem to be exposed in the julia extension.

They can be disabled in VSCode settings.
Copy link
Member

@timholy timholy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking very good to me. Once others who know more about the vscode side are happy with it, I'm fine with merging.

One question: how VSCode-specific is this, vs being LanguageServer-focused? Your PR here justifiably got a mention on #appreciation on the Julia slack, and one of the only concerns was about the tightness of the coupling. I'm not sure of the answer myself.

TypedSyntax/src/show.jl Outdated Show resolved Hide resolved
position is 0-indexed so need to add 1 as JuliaSyntax expects 1-based indexing.
@pfitzseb
Copy link
Member

this doesn't seem to be exposed in the julia extension.

Yeah, true. The focus switching would be super annoying for interactive use.

@Zentrik
Copy link
Collaborator Author

Zentrik commented Aug 14, 2023

One question: how VSCode-specific is this, vs being LanguageServer-focused? Your PR here justifiably got a mention on #appreciation on the Julia slack, and one of the only concerns was about the tightness of the coupling. I'm not sure of the answer myself.

This doesn't use the LanguageServer at all, I don't think its possible to use LanguageServer here as that would require LanguageServer to load and compile user code. The vscode specific functions are in https://github.com/Zentrik/Cthulhu.jl/blob/master/TypedSyntax/src/vscode.jl and they are Base.show and those with vscode in their name. I think the main work to add this to other IDEs would be doing something like julia-vscode/julia-vscode#3328 and the related code that was already in the extension.

[skip ci]
[skip ci]
@Zentrik
Copy link
Collaborator Author

Zentrik commented Aug 19, 2023

I feel like the redundancy of having the whole function printed in the REPL is more annoying than having the editor jump to the function you've descended into -- that workflow matches the normal debugging experience pretty closely. We could maybe add a [F]ollow in editor toggle?

I've got this working, I'll make a pr once this is merged as it's largely separate of this pr and not VSCode specific.

Comment on lines +1 to +2
diagnostics_available_vscode() = isdefined(Main, :VSCodeServer) && Main.VSCodeServer isa Module && isdefined(Main.VSCodeServer, :DIAGNOSTICS_ENABLED) && Main.VSCodeServer.DIAGNOSTICS_ENABLED[]
inlay_hints_available_vscode() = isdefined(Main, :VSCodeServer) && Main.VSCodeServer isa Module && isdefined(Main.VSCodeServer, :INLAY_HINTS_ENABLED) && Main.VSCodeServer.INLAY_HINTS_ENABLED[]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pfitzseb VSCodeServer is not registered right?

Otherwise we could use package extensions here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not registered, yes. Do package extensions actually require that though? VSCodeServer is a package and is loaded as such, so the appropriate hooks should fire.

@vchuravy vchuravy merged commit e7c424e into JuliaDebug:master Aug 20, 2023
@vtjnash
Copy link
Contributor

vtjnash commented Aug 24, 2023

I am getting an error now after Pkg.update from this PR. It seems like the compat bounds are incorrect for these packages now, since I have Cthulhu dev'd but the released version of TypedSyntax, and those are not compatible anymore.

(@v1.8) pkg> st -m Cthulhu TypedSyntax
Status `~/.julia/environments/v1.8/Manifest.toml`
  [f68482b8] Cthulhu v2.9.2 `https://github.com/JuliaDebug/Cthulhu.jl.git#master`
  [d265eb64] TypedSyntax v1.2.1
julia> Base.locate_package(Base.identify_package(Base.identify_package(Main, "Cthulhu"), "TypedSyntax"))
"/home/vtjnash/.julia/packages/TypedSyntax/vbZHi/src/TypedSyntax.jl"

julia> Base.locate_package(Base.identify_package(Main, "Cthulhu"))
"/home/vtjnash/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl"
julia> using Cthulhu                                                                                                                                                                                      
Precompiling Cthulhu                                                                                                                                                                                      
        Info Given Cthulhu was explicitly requested, output will be shown live                                                                                                                            
ERROR: LoadError: UndefVarError: `clear_all_vscode` not defined                                                                                                                                           
Stacktrace:                                                                                                                                                                                               
  [1] getproperty(x::Module, f::Symbol)                                                                                                                                                                   
    @ Base ./Base.jl:31                                                                                                                                                                                   
  [2] __descend_with_error_handling(args::Any; terminal::Any, kwargs...)                                                                                                                                  
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:213                                                                                                                                          
  [3] _descend_with_error_handling(f::Any, argtypes::Any; kwargs::@Kwargs{iswarn::Bool, terminal::REPL.Terminals.TTYTerminal})                                                                            
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:200                                                                                                                                          
  [4] descend_code_typed(::Any, ::Vararg{Any}; kwargs::@Kwargs{terminal::REPL.Terminals.TTYTerminal})                                                                                                     
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:158                                                                                                                                          
  [5] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:831 [inlined]                                                                                                                                        
  [6] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/PrecompileTools/kmH5L/src/workloads.jl:78 [inlined]                                                                                                                               
  [7] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:830 [inlined]                                   
  [8] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/PrecompileTools/kmH5L/src/workloads.jl:140 [inlined]                                                                                                                              
  [9] top-level scope                                                                                                                                                                                     
    @ ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:826                                                                                                                                                  
 [10] include                                                                                                                                                                                             
    @ Base ./Base.jl:494 [inlined]                                                                                                                                                                        
 [11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, so
urce::Nothing)                                                                                                                                                                                            
    @ Base ./loading.jl:2216                                                                         
 [12] top-level scope                                                                                                                                                                                     
    @ stdin:3                                                                                                                                                                                             
in expression starting at /home/vtjnash/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:1                                                                                                                    
in expression starting at stdin:3                                                                                                                                                                         
                                                                                                                                                                                                          
caused by: UndefVarError: `diagnostics_available_vscode` not defined                                                                                                                                      
Stacktrace:                                                                                                                                                                                               
  [1] getproperty(x::Module, f::Symbol)                                                                                                                                                                   
    @ Base ./Base.jl:31                                                                              
  [2] cthulhu_typed(io::IOContext{IOBuffer}, debuginfo::Symbol, src::Core.CodeInfo, rt::Any, effects::Core.Compiler.Effects, mi::Core.MethodInstance; iswarn::Bool, hide_type_stable::Bool, optimize::Bool
, pc2remarks::Nothing, pc2effects::Nothing, inline_cost::Bool, type_annotations::Bool, annotate_source::Bool, inlay_types_vscode::Bool, diagnostics_vscode::Bool, interp::Cthulhu.CthulhuInterpreter)
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/codeview.jl:152                                                                                                                                         
  [3] cthulhu_typed(::IOContext{IOBuffer}, ::Cthulhu.DInfo.DebugInfo, ::Core.CodeInfo, ::Vararg{Any}; kwargs::@Kwargs{iswarn::Bool, optimize::Bool, hide_type_stable::Bool, pc2remarks::Nothing, pc2effect
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/codeview.jl:121                                                                                                                                         
  [4] (::Cthulhu.var"#92#99"{Cthulhu.CthulhuInterpreter, Cthulhu.DInfo.DebugInfo, Core.CodeInfo, Core.CodeInfo, DataType, Core.MethodInstance, Bool, Bool, Nothing, Nothing, Bool, Bool})(lambda_io::IOCon
text{IOBuffer})                                                                                                                                                                                           
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:481                                                                                                                                          
  [5] stringify(f::Any, context::IOContext{Base.DevNull})                                                                                                                                                 
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/ui.jl:90                                                                                                                                                
  [6] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, curs::Cthulhu.CthulhuCursor; override::Nothing, debuginfo::Symbol, optimize::Bool, interruptexc::Bool, iswarn::Bool, 
hide_type_stable::Bool, verbose::Nothing, remarks::Bool, with_effects::Bool, inline_cost::Bool, type_annotations::Bool, annotate_source::Bool, inlay_types_vscode::Bool, diagnostics_vscode::Bool)        
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:480                                                                                                                                          
  [7] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, mi::Core.MethodInstance; kwargs::@Kwargs{iswarn::Bool})                                                              
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:739                                                                                                                                          
  [8] _descend(term::REPL.Terminals.TTYTerminal, args::Any; interp::Core.Compiler.NativeInterpreter, kwargs::@Kwargs{iswarn::Bool})                                                                       
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:755                                                                                                                                          
  [9] __descend_with_error_handling(args::Any; terminal::Any, kwargs...)                                                                                                                                  
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:211                                                                                                                                          
 [10] _descend_with_error_handling(f::Any, argtypes::Any; kwargs::@Kwargs{iswarn::Bool, terminal::REPL.Terminals.TTYTerminal})                                                                            
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:200                                                                                                                                          
 [11] descend_code_typed(::Any, ::Vararg{Any}; kwargs::@Kwargs{terminal::REPL.Terminals.TTYTerminal})                                                                                                     
    @ Cthulhu ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:158                                                                                                                                          
 [12] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:831 [inlined]                                                                                                                                        
 [13] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/PrecompileTools/kmH5L/src/workloads.jl:78 [inlined]                                                                                                                               
 [14] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:830 [inlined]                                                                                                                                        
 [15] macro expansion                                                                                                                                                                                     
    @ ~/.julia/packages/PrecompileTools/kmH5L/src/workloads.jl:140 [inlined]                                                                                                                              
 [16] top-level scope                                                                                                                                                                                     
    @ ~/.julia/packages/Cthulhu/stXnv/src/Cthulhu.jl:826                                                                                                                                                  
 [17] include                                                                                                                                                                                             
    @ Base ./Base.jl:494 [inlined]                                                                                                                                                                        
 [18] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, so
urce::Nothing)                                                                                                                                                                                            
    @ Base ./loading.jl:2216                                                                                                                                                                              
 [19] top-level scope                                                                                                                                                                                     
    @ stdin:3                                                                                        

Comment on lines +56 to +85
module TestVSCodeExt # stops modules defined in test files from overwriting stuff from previous test
using Test, PerformanceTestTools, ..VSCodeServer
@testset "runtests.jl VSCodeExt" begin
@testset "test_Cthulhu.jl" begin
include("test_Cthulhu.jl")
end

@testset "test_codeview.jl" begin
include("test_codeview.jl")
include("test_codeview_vscode.jl")
end

# TODO enable this test on nightly
if false
@testset "test_irshow.jl" begin
include("test_irshow.jl")
end
else
@info "skipped test_irshow.jl"
end

@testset "test_terminal.jl" begin
include("test_terminal.jl")
end

@testset "test_AbstractInterpreter.jl" begin
include("test_AbstractInterpreter.jl")
end
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Zentrik What's the whole purpose of duplicating tests here? Can't we include("test_codeview_vscode.jl") only?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's some additional work that Cthulhu does when running under VSCode that I wanted to test in more cases than "test_codeview_vscode.jl". The main thing I was worried about was descending into all callees here.
I don't think it's particularly important to run these additional tests given the code has been working for a while, so if you want you can nix it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, it sounds like the purpose of running these duplicated tests was to verify that descending into all callees doesn't lead to any timing issues, right? If so, we might consider removing them. Alternatively, if we decide to keep them, it could be a better design choice to extract the main portion of runtests.jl into a separate file and then include that file in both runtests.jl and test_VSCode.jl.

@el-oso
Copy link

el-oso commented Aug 28, 2024

I see that this was merged into master, how can I test it myself?

@Zentrik
Copy link
Collaborator Author

Zentrik commented Aug 28, 2024

Call Cthulhu.@descend ... in the VSCode Julia REPL and it should just work.

@el-oso
Copy link

el-oso commented Aug 28, 2024

This is very cool! thanks!
Side question: Does this work in nvim? :-)

@Zentrik
Copy link
Collaborator Author

Zentrik commented Aug 28, 2024

No VSCode only unfortunately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants