Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emit line info for generator variants #73460

Merged
merged 9 commits into from
Jun 26, 2020
Merged

Conversation

tmandry
Copy link
Member

@tmandry tmandry commented Jun 17, 2020

Debuggers should be able to read a generator / async fn state machine and show the line it's suspended at. Eventually, this could grow into an "async stack trace" feature of sorts. While no debugger support this for Rust today, this PR adds the debuginfo necessary for that support to exist.

This gist shows the resulting debuginfo for a simple example. Here's a snippet:

0x00000986:           DW_TAG_variant
                        DW_AT_discr_value       (0x03)

0x00000988:             DW_TAG_member
                          DW_AT_name    ("3")
                          DW_AT_type    (0x000009bc "Suspend0")
                          DW_AT_decl_file       ("/home/tmandry/code/playground/generator-simple.rs")
                          DW_AT_decl_line       (6)
                          DW_AT_alignment       (8)
                          DW_AT_data_member_location    (0x00)

The file and line have been added here. The line currently points to the beginning of the statement containing the yield (or await), because that's what the MIR source info points to for the yield terminator. (We may want to point to the yield or await line specifically, but that can be done independently of this change.)

Debuggers don't know how to use this kind of info yet. However, we're hoping to experiment with adding such support to Fuchsia's debugger. It would be exciting if someone were interested in adding similar to support to gdb/lldb.

r? @oli-obk
cc @eddyb @jonas-schievink

Part of #73524.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 17, 2020
@tmiasko
Copy link
Contributor

tmiasko commented Jun 18, 2020

This could be tested by looking at generated LLVM IR metadata. There are a few debuginfo tests in src/test/codegen already, but clang/test/CodeGen might be more useful as a source of inspiration.

@tmandry
Copy link
Member Author

tmandry commented Jun 18, 2020

r? @ghost while I make some changes

@tmandry tmandry added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 18, 2020
@rust-highfive
Copy link
Collaborator

The job mingw-check of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
##[section]Starting: Linux mingw-check
##[section]Starting: Initialize job
Agent name: 'Azure Pipelines 7'
Agent machine name: 'fv-az578'
Current agent version: '2.170.1'
##[group]Operating System
16.04.6
LTS
LTS
##[endgroup]
##[group]Virtual Environment
Environment: ubuntu-16.04
Version: 20200604.1
Included Software: https://github.com/actions/virtual-environments/blob/ubuntu16/20200604.1/images/linux/Ubuntu1604-README.md
##[endgroup]
Agent running as: 'vsts'
Prepare build directory.
Set build variables.
Download all required tasks.
Download all required tasks.
Downloading task: Bash (3.163.3)
Checking job knob settings.
   Knob: AgentToolsDirectory = /opt/hostedtoolcache Source: ${AGENT_TOOLSDIRECTORY} 
   Knob: AgentPerflog = /home/vsts/perflog Source: ${VSTS_AGENT_PERFLOG} 
Start tracking orphan processes.
##[section]Finishing: Initialize job
##[section]Starting: Configure Job Name
==============================================================================
---
========================== Starting Command Output ===========================
[command]/bin/bash --noprofile --norc /home/vsts/work/_temp/25934668-ae0a-4ad5-a8ba-fca20496d4eb.sh

##[section]Finishing: Disable git automatic line ending conversion
##[section]Starting: Checkout rust-lang/rust@refs/pull/73460/merge to s
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
---
##[command]git remote add origin https://github.com/rust-lang/rust
##[command]git config gc.auto 0
##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
##[command]git config --get-all http.proxy
##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/73460/merge:refs/remotes/pull/73460/merge
---
Step 6/7 : ENV RUN_CHECK_WITH_PARALLEL_QUERIES 1
 ---> Running in a297404f768d
Removing intermediate container a297404f768d
 ---> 755c587daf39
Step 7/7 : ENV SCRIPT python3 ../x.py test src/tools/expand-yaml-anchors &&            python3 ../x.py check --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu &&            python3 ../x.py build --stage 0 src/tools/build-manifest &&            python3 ../x.py test --stage 0 src/tools/compiletest &&            python3 ../x.py test src/tools/tidy &&            python3 ../x.py doc --stage 0 src/libstd &&            /scripts/validate-toolstate.sh
Removing intermediate container 8a4e6c84911e
 ---> 460b50c8a254
Successfully built 460b50c8a254
Successfully tagged rust-ci:latest
Successfully tagged rust-ci:latest
Built container sha256:460b50c8a25485448f839d16d2c136cf0ac76ed1adf72ec75f6bd2a353908a53
Uploading finished image to https://ci-caches.rust-lang.org/docker/d3f0b748e51c65823b26e5c9aa3312fcd51003139be9a43290b3a342a55aa1a19cca0eefe8cd2813f66ed4bd2ebafe54507cc7aa11fb4b3d0cb6de76d4374fee
upload failed: - to s3://rust-lang-ci-sccache2/docker/d3f0b748e51c65823b26e5c9aa3312fcd51003139be9a43290b3a342a55aa1a19cca0eefe8cd2813f66ed4bd2ebafe54507cc7aa11fb4b3d0cb6de76d4374fee An error occurred (InvalidAccessKeyId) when calling the CreateMultipartUpload operation: The AWS Access Key Id you provided does not exist in our records.
[CI_JOB_NAME=mingw-check]
== clock drift check ==
  local time: Wed Jun 17 23:51:47 UTC 2020
  network time: Wed, 17 Jun 2020 23:51:47 GMT
---
    Checking chalk-rust-ir v0.10.0
    Checking rustc_ast_pretty v0.0.0 (/checkout/src/librustc_ast_pretty)
    Checking rustc_hir v0.0.0 (/checkout/src/librustc_hir)
    Checking rustc_session v0.0.0 (/checkout/src/librustc_session)
    Checking rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
    Checking rustc_parse v0.0.0 (/checkout/src/librustc_parse)
    Checking rustc_hir_pretty v0.0.0 (/checkout/src/librustc_hir_pretty)
    Checking rustc_ast_lowering v0.0.0 (/checkout/src/librustc_ast_lowering)
    Checking rustc_ast_passes v0.0.0 (/checkout/src/librustc_ast_passes)
---
configure: rust.verify-llvm-ir  := True
configure: rust.debug-assertions := True
configure: build.cargo-native-static := True
configure: llvm.assertions      := True
configure: build.configure-args := ['--enable-sccache', '--disable-manage-submodu ...
configure: writing `config.toml` in current directory
configure: 
configure: run `python /checkout/x.py --help`
configure: 
---
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model  : 85
model name : Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
microcode : 0xffffffff
cpu MHz  : 2593.904
cache size : 36608 KB
physical id : 0
---
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model  : 85
model name : Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
microcode : 0xffffffff
cpu MHz  : 2593.904
cache size : 36608 KB
physical id : 0
---
Hugepagesize:       2048 kB
DirectMap4k:      128960 kB
DirectMap2M:     4065280 kB
DirectMap1G:     5242880 kB
+ python3 ../x.py test src/tools/expand-yaml-anchors
Ensuring the YAML anchors in the GitHub Actions config were expanded
Ensuring the YAML anchors in the GitHub Actions config were expanded
Building stage0 tool expand-yaml-anchors (x86_64-unknown-linux-gnu)
   Compiling unicode-xid v0.2.0
   Compiling syn v1.0.11
   Compiling linked-hash-map v0.5.2
   Compiling lazy_static v1.4.0
   Compiling lazy_static v1.4.0
   Compiling yaml-rust v0.4.3
   Compiling quote v1.0.2
   Compiling thiserror-impl v1.0.5
   Compiling thiserror v1.0.5
   Compiling yaml-merge-keys v0.4.0
   Compiling expand-yaml-anchors v0.1.0 (/checkout/src/tools/expand-yaml-anchors)
Build completed successfully in 0:00:23
+ python3 ../x.py check --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu
    Finished dev [unoptimized] target(s) in 0.15s
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> i686-pc-windows-gnu)
---
    Checking chalk-rust-ir v0.10.0
    Checking rustc_ast_pretty v0.0.0 (/checkout/src/librustc_ast_pretty)
    Checking rustc_hir v0.0.0 (/checkout/src/librustc_hir)
    Checking rustc_session v0.0.0 (/checkout/src/librustc_session)
    Checking rustc_query_system v0.0.0 (/checkout/src/librustc_query_system)
    Checking rustc_parse v0.0.0 (/checkout/src/librustc_parse)
    Checking rustc_hir_pretty v0.0.0 (/checkout/src/librustc_hir_pretty)
    Checking rustc_ast_lowering v0.0.0 (/checkout/src/librustc_ast_lowering)
    Checking rustc_ast_passes v0.0.0 (/checkout/src/librustc_ast_passes)
---
skip untracked path cpu-usage.csv during rustfmt invocations
skip untracked path src/doc/book/ during rustfmt invocations
skip untracked path src/doc/rust-by-example/ during rustfmt invocations
skip untracked path src/llvm-project/ during rustfmt invocations
Diff in /checkout/src/librustc_mir/transform/generator.rs at line 771:
         SourceInfo::outermost(body_span.shrink_to_lo()),
         SourceInfo::outermost(body_span.shrink_to_hi()),
         SourceInfo::outermost(body_span.shrink_to_hi()),
-    ].iter().copied().collect();
+    .iter()
+    .copied()
+    .collect();
 
 
     // Build the generator variant field list.
     // Create a map from local indices to generator struct indices.
Running `"/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/rustfmt" "--config-path" "/checkout" "--edition" "2018" "--unstable-features" "--skip-children" "--check" "/checkout/src/librustc_mir/transform/generator.rs"` failed.
If you're running `tidy`, try again with `--bless`. Or, if you just want to format code, run `./x.py fmt` instead.
Build completed unsuccessfully in 0:00:31
== clock drift check ==
  local time: Thu Jun 18 00:05:56 UTC 2020
  network time: Thu, 18 Jun 2020 00:05:56 GMT
  network time: Thu, 18 Jun 2020 00:05:56 GMT
== end clock drift check ==

##[error]Bash exited with code '1'.
##[section]Finishing: Run build
##[section]Starting: Checkout rust-lang/rust@refs/pull/73460/merge to s
Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version      : 1.0.0
Author       : Microsoft
Author       : Microsoft
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
==============================================================================
Cleaning any cached credential from repository: rust-lang/rust (GitHub)
##[section]Finishing: Checkout rust-lang/rust@refs/pull/73460/merge to s
Cleaning up task key
Start cleaning up orphan processes.
Terminate orphan process: pid (5417) (python)
##[section]Finishing: Finalize Job

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

@tmandry tmandry marked this pull request as draft June 18, 2020 19:14
@tmandry tmandry force-pushed the variant-lineinfo branch 2 times, most recently from 735b3cc to dd883ed Compare June 18, 2020 23:37
@tmandry tmandry marked this pull request as ready for review June 19, 2020 00:35
@tmandry
Copy link
Member Author

tmandry commented Jun 19, 2020

Tests added as suggested by @tmiasko. Verified that this works locally on Mac and Windows, too.

r? @oli-obk

I'm looking for someone who has touched debuginfo before, feel free to pick another reviewer. :)

@@ -1,5 +1,5 @@
// MIR for `main::{{closure}}#0` 0 generator_drop
// generator_layout = GeneratorLayout { field_tys: [std::string::String], variant_fields: [[], [], [], [_0]], storage_conflicts: BitMatrix { num_rows: 1, num_columns: 1, words: [1], marker: PhantomData } }
// generator_layout = GeneratorLayout { field_tys: [std::string::String], variant_fields: [[], [], [], [_0]], variant_source_info: [SourceInfo { span: $DIR/generator-drop-cleanup.rs:10:15: 10:15 (#0), scope: scope[0] }, SourceInfo { span: $DIR/generator-drop-cleanup.rs:13:6: 13:6 (#0), scope: scope[0] }, SourceInfo { span: $DIR/generator-drop-cleanup.rs:13:6: 13:6 (#0), scope: scope[0] }, SourceInfo { span: $DIR/generator-drop-cleanup.rs:12:9: 12:14 (#0), scope: scope[1] }], storage_conflicts: BitMatrix { num_rows: 1, num_columns: 1, words: [1], marker: PhantomData } }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preexisting, but made worse by this PR: we should do a human readable printing of the layout

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I.. might have gone a bit overboard on this one. Let me know what you think :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it

@oli-obk
Copy link
Contributor

oli-obk commented Jun 19, 2020

The impl changes make sense to me, though I can't judge if this is enough to help debuggers make use of it. Please open a tracking issue for this topic so we have something tracking the status and future plans.

@tmandry
Copy link
Member Author

tmandry commented Jun 20, 2020

Please open a tracking issue for this topic so we have something tracking the status and future plans.

Done: #73524

@tmandry tmandry added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 20, 2020
@oli-obk
Copy link
Contributor

oli-obk commented Jun 20, 2020

@bors r+

@bors
Copy link
Contributor

bors commented Jun 20, 2020

📌 Commit 5fbb6ae has been approved by oli-obk

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 20, 2020
RalfJung added a commit to RalfJung/rust that referenced this pull request Jun 20, 2020
Emit line info for generator variants

Debuggers should be able to read a generator / async fn state machine and show the line it's suspended at. Eventually, this could grow into an "async stack trace" feature of sorts. While no debugger support this for Rust today, this PR adds the debuginfo necessary for that support to exist.

[This gist](https://gist.github.com/tmandry/6d7004fa008684f76809208847459f9b) shows the resulting debuginfo for a simple example. Here's a snippet:

```
0x00000986:           DW_TAG_variant
                        DW_AT_discr_value       (0x03)

0x00000988:             DW_TAG_member
                          DW_AT_name    ("3")
                          DW_AT_type    (0x000009bc "Suspend0")
                          DW_AT_decl_file       ("/home/tmandry/code/playground/generator-simple.rs")
                          DW_AT_decl_line       (6)
                          DW_AT_alignment       (8)
                          DW_AT_data_member_location    (0x00)
```

The file and line have been added here. The line currently points to the beginning of the statement containing the yield (or await), because that's what the MIR source info points to for the yield terminator. (We may want to point to the yield or await line specifically, but that can be done independently of this change.)

Debuggers don't know how to use this kind of info yet. However, we're hoping to experiment with adding such support to Fuchsia's debugger. It would be exciting if someone were interested in adding similar to support to gdb/lldb.

r? @oli-obk
cc @eddyb @jonas-schievink

Part of rust-lang#73524.
@bors bors removed the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Jun 20, 2020
@bors bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jun 20, 2020
@tmandry tmandry force-pushed the variant-lineinfo branch 3 times, most recently from 49a3457 to 0c523e3 Compare June 24, 2020 21:25
@tmandry
Copy link
Member Author

tmandry commented Jun 24, 2020

Rebased and got tests working.

I've found using FileCheck on LLVM IR to be quite brittle for debuginfo. It's really impossible to write good tests that are resilient to trivial reorderings. We should consider support for checking the output of llvm-dwarfdump, which is a lot more stable.

In the meantime I've split out the generator and async fn into separate tests, which helps, but still doesn't let me test everything I want.

..which doesn't use variant types.
This keeps FileCheck from tripping over unimportant differences in
codegen.
@tmandry
Copy link
Member Author

tmandry commented Jun 25, 2020

@bors r=oli-obk

@bors
Copy link
Contributor

bors commented Jun 25, 2020

📌 Commit 887fbd9 has been approved by oli-obk

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 25, 2020
Manishearth added a commit to Manishearth/rust that referenced this pull request Jun 26, 2020
Emit line info for generator variants

Debuggers should be able to read a generator / async fn state machine and show the line it's suspended at. Eventually, this could grow into an "async stack trace" feature of sorts. While no debugger support this for Rust today, this PR adds the debuginfo necessary for that support to exist.

[This gist](https://gist.github.com/tmandry/6d7004fa008684f76809208847459f9b) shows the resulting debuginfo for a simple example. Here's a snippet:

```
0x00000986:           DW_TAG_variant
                        DW_AT_discr_value       (0x03)

0x00000988:             DW_TAG_member
                          DW_AT_name    ("3")
                          DW_AT_type    (0x000009bc "Suspend0")
                          DW_AT_decl_file       ("/home/tmandry/code/playground/generator-simple.rs")
                          DW_AT_decl_line       (6)
                          DW_AT_alignment       (8)
                          DW_AT_data_member_location    (0x00)
```

The file and line have been added here. The line currently points to the beginning of the statement containing the yield (or await), because that's what the MIR source info points to for the yield terminator. (We may want to point to the yield or await line specifically, but that can be done independently of this change.)

Debuggers don't know how to use this kind of info yet. However, we're hoping to experiment with adding such support to Fuchsia's debugger. It would be exciting if someone were interested in adding similar to support to gdb/lldb.

r? @oli-obk
cc @eddyb @jonas-schievink

Part of rust-lang#73524.
bors added a commit to rust-lang-ci/rust that referenced this pull request Jun 26, 2020
…arth

Rollup of 14 pull requests

Successful merges:

 - rust-lang#72617 (Add a fast path for `std::thread::panicking`.)
 - rust-lang#72738 (Self contained linking option)
 - rust-lang#72770 (Implement mixed script confusable lint.)
 - rust-lang#73418 (Add unstable `core::mem::variant_count` intrinsic)
 - rust-lang#73460 (Emit line info for generator variants)
 - rust-lang#73534 (Provide suggestions for some moved value errors)
 - rust-lang#73538 (make commented examples use valid syntax, and be more consistent )
 - rust-lang#73581 (Create 0766 error code)
 - rust-lang#73619 (Document the mod keyword)
 - rust-lang#73621 (Document the mut keyword)
 - rust-lang#73648 (Document the return keyword)
 - rust-lang#73673 (Fix ptr doc warnings.)
 - rust-lang#73674 (Tweak binop errors)
 - rust-lang#73687 (Clean up E0701 explanation)

Failed merges:

 - rust-lang#73708 (Explain move errors that occur due to method calls involving `self` (take two))

r? @ghost
@bors
Copy link
Contributor

bors commented Jun 26, 2020

☔ The latest upstream changes (presumably #73746) made this pull request unmergeable. Please resolve the merge conflicts.

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 26, 2020
@bors bors merged commit 3f5b8c8 into rust-lang:master Jun 26, 2020
@tmandry tmandry deleted the variant-lineinfo branch June 27, 2020 00:21
@cuviper cuviper added this to the 1.46 milestone May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants