Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Profile: make heap snapshots viewable in vscode viewer #53833

Merged
merged 3 commits into from Apr 5, 2024

Conversation

IanButterworth
Copy link
Sponsor Member

@IanButterworth IanButterworth commented Mar 24, 2024

Tested against https://github.com/microsoft/vscode-v8-heap-tools/tree/main/v8-heap-parser by copying a julia heapsnapshot into the test dir as basic.heapsnapshot and running cargo test

Update: This is working, except for the graph view, which appears to be an upstream issue microsoft/vscode-js-profile-visualizer#175


Original issues

With these changes, it still fails, but because of the middle line here

"codelocs",
"�\u0003",
"<generic memory - malloc>",

with

failures:

---- decoder::tests::test_basic_heapsnapshot stdout ----
thread 'decoder::tests::test_basic_heapsnapshot' panicked at v8-heap-parser/src/decoder.rs:574:33:
expect no errors: Error("invalid unicode code point", line: 6020770, column: 9)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- graph::tests::test_retained_sizes stdout ----
thread 'graph::tests::test_retained_sizes' panicked at v8-heap-parser/src/graph.rs:719:45:
expect no errors: Error("invalid unicode code point", line: 6020770, column: 9)


failures:
    decoder::tests::test_basic_heapsnapshot
    graph::tests::test_retained_sizes

If I manually empty that string the only test that fails is a content comparison against a fixed reference.
Does the string sanitizer need fixing or is this some other issue?

However.. the string-fixed file loads in neither chrome devtools, nor vscode's viewer.

I'm opening this in case you see the issue, @vilterp @NHDaly

@vtjnash
Copy link
Sponsor Member

vtjnash commented Mar 24, 2024

JavaScript JSON strings differ slightly from Unicode encoding in that high code points are written as malformed UTF-16 surrogate pairs (when encoded as \x or \u escape sequences) instead of being encoded as their Unicode value (which is only supported in the very latest new version of ecmascript). Julia also may include arbitrary binary data in a string, while JSON only permits valid UTF-8 sequences (even as escape sequences)

@IanButterworth
Copy link
Sponsor Member Author

IanButterworth commented Mar 24, 2024

I see. I just coincidentally pushed a change to only print unicode assigned chars. Does that sound reasonable?

With this they open in vscode now, although the graph view either takes longer than I've tried to build, or hangs. It's not clear

Screenshot 2024-03-24 at 11 48 27 AM

Screenshot 2024-03-24 at 11 50 04 AM

@IanButterworth
Copy link
Sponsor Member Author

Note that vscode is on an older version of https://github.com/microsoft/vscode-v8-heap-tools/tree/1e1cbd86d9376995e459c31fbaacbd0f7a03adb9
Testing on the right commit helped..

@IanButterworth IanButterworth marked this pull request as ready for review March 24, 2024 19:40
@IanButterworth IanButterworth added backport 1.10 Change should be backported to the 1.10 release backport 1.11 Change should be backported to release-1.11 labels Mar 24, 2024
This was referenced Mar 27, 2024
@IanButterworth
Copy link
Sponsor Member Author

It turns out that opening a heapsnapshot in the graph viewer via the dialog isn't valid and will be fixed (disabled) in microsoft/vscode-js-profile-visualizer#179

Instead, clicking on the graph icon where presented in the table view beside an object opens the graph successfully.

Screenshot 2024-04-04 at 4 30 50 PM

@vilterp @NHDaly I'll merge this tomorrow if I don't hear otherwise.

@vilterp
Copy link
Contributor

vilterp commented Apr 4, 2024

Oh, so it's just a 'retainer graph'? i.e. starting from a given object, go up to the roots? that's cool

Copy link
Contributor

@vilterp vilterp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@IanButterworth
Copy link
Sponsor Member Author

Just remembered I needed to update stdlib/Manifest.toml because Profile took on Unicode as a dep.

@IanButterworth IanButterworth added the status:merge me PR is reviewed. Merge when all tests are passing label Apr 4, 2024
@IanButterworth IanButterworth merged commit 57bbff6 into JuliaLang:master Apr 5, 2024
8 checks passed
@IanButterworth IanButterworth deleted the ib/heap_vscode branch April 5, 2024 02:00
@IanButterworth IanButterworth removed the status:merge me PR is reviewed. Merge when all tests are passing label Apr 5, 2024
KristofferC pushed a commit that referenced this pull request Apr 9, 2024
KristofferC added a commit that referenced this pull request Apr 9, 2024
Backported PRs:
- [x] #53757 <!-- Add an IndexStyle example to the diagind docstring -->
- [x] #53809 <!-- Add missing GC_POP() in emit_cfunction -->
- [x] #53789 <!-- also check that UUID of project is non-null when
treating it as a package -->
- [x] #53805 <!-- precompilepkgs: simplify custom config printing if
only one -->
- [x] #53822 <!-- Bump libuv -->
- [x] #53837 <!-- update MPFR to 4.2.1 -->
- [x] #53862 <!-- precompilepkgs: fix error reporting -->
- [x] #53774 <!-- Remove some duplicates from emitted compilation traces
-->
- [ ] #53696 <!-- add invokelatest to on_done callback in bracketed
paste -->
- [x] #53383 <!-- Add `_unsetindex!` methods for `SubArray`s and
`CartesianIndex`es -->
- [x] #53475 <!-- Fix boundscheck in unsetindex for SubArrays -->
- [x] #53888 
- [x] #53870 <!-- Revert change to checksum for llvm-julia -->
- [x] #53906 <!-- Add `Base.isrelocatable(pkg)` -->
- [x] #53833 <!-- Profile: make heap snapshots viewable in vscode viewer
-->
- [x] #53961 <!-- `LazyString` in `LinearAlgebra.checksquare` error
message -->
- [x] #53962 <!-- Use StringMemory instead of StringVector where
possible -->
- [x] #53825 <!-- profile: doc: update the `Allocs.@profile` doc string
-->
- [x] #53975 <!-- `LazyString` in `DimensionMismatch` error messages in
broadcasting -->
- [x] #53905 <!-- Avoid repeated precompilation when loading from
non-relocatable cachefiles -->
- [x] #53896 <!-- Make reshape and view on Memory produce Arrays and
delete wrap -->
- [x] #53991 <!-- Test and fix non-int-length bug in `view(::Memory,
::Union{UnitRange, Base.OneTo})` -->
vtjnash pushed a commit that referenced this pull request Apr 16, 2024
Followup to #53833
Fixes a failure seen in #53974
(below)

I believe this is the more correct check to make?

The heapsnapshot generated from this PR is viewable in vscode.

```
2024-04-06 09:33:58 EDT	      From worker 7:	ERROR: Base.InvalidCharError{Char}('\xc1\xae')
2024-04-06 09:33:58 EDT	      From worker 7:	Stacktrace:
2024-04-06 09:33:58 EDT	      From worker 7:	  [1] throw_invalid_char(c::Char)
2024-04-06 09:33:58 EDT	      From worker 7:	    @ Base ./char.jl:86
2024-04-06 09:33:58 EDT	      From worker 7:	  [2] UInt32
2024-04-06 09:33:58 EDT	      From worker 7:	    @ ./char.jl:133 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [3] category_code
2024-04-06 09:33:58 EDT	      From worker 7:	    @ ./strings/unicode.jl:339 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [4] isassigned
2024-04-06 09:33:58 EDT	      From worker 7:	    @ ./strings/unicode.jl:355 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [5] isassigned
2024-04-06 09:33:58 EDT	      From worker 7:	    @ /cache/build/tester-amdci5-14/julialang/julia-master/julia-41d026beaf/share/julia/stdlib/v1.12/Unicode/src/Unicode.jl:138 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [6] print_str_escape_json(stream::IOStream, s::String)
2024-04-06 09:33:58 EDT	      From worker 7:	    @ Profile.HeapSnapshot /cache/build/tester-amdci5-14/julialang/julia-master/julia-41d026beaf/share/julia/stdlib/v1.12/Profile/src/heapsnapshot_reassemble.jl:239
2024-04-06 09:33:59 EDT	      From worker 7:	  [7] (::Profile.HeapSnapshot.var"#5#6"{IOStream})(strings_io::IOStream)
2024-04-06 09:33:59 EDT	      From worker 7:	    @ Profile.HeapSnapshot /cache/build/tester-amdci5-14/julialang/julia-master/julia-41d026beaf/share/julia/stdlib/v1.12/Profile/src/heapsnapshot_reassemble.jl:192
```
@vtjnash
Copy link
Sponsor Member

vtjnash commented Apr 16, 2024

I had thought jl_static_show_string from #52799 would have already fixed this, but I didn't realize that this code used a different string encoder that is somewhat more specific to JSON output. That change might still affect the places where jl_static_show is used here (e.g. for printing some Type representations) but not commonly so

KristofferC pushed a commit that referenced this pull request Apr 17, 2024
Followup to #53833
Fixes a failure seen in #53974
(below)

I believe this is the more correct check to make?

The heapsnapshot generated from this PR is viewable in vscode.

```
2024-04-06 09:33:58 EDT	      From worker 7:	ERROR: Base.InvalidCharError{Char}('\xc1\xae')
2024-04-06 09:33:58 EDT	      From worker 7:	Stacktrace:
2024-04-06 09:33:58 EDT	      From worker 7:	  [1] throw_invalid_char(c::Char)
2024-04-06 09:33:58 EDT	      From worker 7:	    @ Base ./char.jl:86
2024-04-06 09:33:58 EDT	      From worker 7:	  [2] UInt32
2024-04-06 09:33:58 EDT	      From worker 7:	    @ ./char.jl:133 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [3] category_code
2024-04-06 09:33:58 EDT	      From worker 7:	    @ ./strings/unicode.jl:339 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [4] isassigned
2024-04-06 09:33:58 EDT	      From worker 7:	    @ ./strings/unicode.jl:355 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [5] isassigned
2024-04-06 09:33:58 EDT	      From worker 7:	    @ /cache/build/tester-amdci5-14/julialang/julia-master/julia-41d026beaf/share/julia/stdlib/v1.12/Unicode/src/Unicode.jl:138 [inlined]
2024-04-06 09:33:58 EDT	      From worker 7:	  [6] print_str_escape_json(stream::IOStream, s::String)
2024-04-06 09:33:58 EDT	      From worker 7:	    @ Profile.HeapSnapshot /cache/build/tester-amdci5-14/julialang/julia-master/julia-41d026beaf/share/julia/stdlib/v1.12/Profile/src/heapsnapshot_reassemble.jl:239
2024-04-06 09:33:59 EDT	      From worker 7:	  [7] (::Profile.HeapSnapshot.var"#5#6"{IOStream})(strings_io::IOStream)
2024-04-06 09:33:59 EDT	      From worker 7:	    @ Profile.HeapSnapshot /cache/build/tester-amdci5-14/julialang/julia-master/julia-41d026beaf/share/julia/stdlib/v1.12/Profile/src/heapsnapshot_reassemble.jl:192
```

(cherry picked from commit c557636)
@KristofferC KristofferC removed the backport 1.11 Change should be backported to release-1.11 label Apr 17, 2024
@KristofferC KristofferC mentioned this pull request May 8, 2024
17 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.10 Change should be backported to the 1.10 release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Heap snapshots aren't viewable in vscode viewer
4 participants