Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Printing AST in custom callback crashes compiler for stolen value #123163

Open
El-Naizin opened this issue Mar 28, 2024 · 4 comments
Open

Printing AST in custom callback crashes compiler for stolen value #123163

El-Naizin opened this issue Mar 28, 2024 · 4 comments
Labels
A-driver Area: rustc_driver that ties everything together into the `rustc` compiler C-discussion Category: Discussion or questions that doesn't represent real issues. requires-nightly This issue requires a nightly compiler in some way. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@El-Naizin
Copy link

Hello,
I was trying to print and modify the Abstract syntax tree during the after_expansion callback.
When trying to print the AST during that callback, the compiler crashes for stolen value.

This is my first error report so let me know if I did anything wrong or am missing anything here

Code

My only file is main.rs:

#![feature(rustc_private)]

extern crate rustc_ast;
extern crate rustc_data_structures;
extern crate rustc_driver;
extern crate rustc_interface;
extern crate rustc_session;

use rustc_ast::Crate;
use rustc_data_structures::profiling::get_resident_set_size;
use rustc_driver::{args, catch_with_exit_code, install_ice_hook, DEFAULT_BUG_REPORT_URL};
use rustc_driver::{init_rustc_env_logger, RunCompiler};
use rustc_driver::{Callbacks, Compilation};
use rustc_session::config::ErrorOutputType;
use rustc_session::EarlyDiagCtxt;
use std::process;
use std::time::Instant;

pub struct AddCallbacks {}

impl Callbacks for AddCallbacks {
    fn after_expansion<'tcx>(
        &mut self,
        _compiler: &rustc_interface::interface::Compiler,
        _queries: &'tcx rustc_interface::Queries<'tcx>,
    ) -> Compilation {
        let krate = _queries.parse().expect("Query couldn't be parsed?");
        let b: &Crate = &(*krate).borrow();
        println!("{b:#?}");
        Compilation::Continue
    }
}

// Main code got from rustc_driver_impl/src/lib.rs/fn main()
// And modified to add custom callbacks
fn main() {
    let _start_time = Instant::now();
    let _start_rss = get_resident_set_size();
    let early_dcx = EarlyDiagCtxt::new(ErrorOutputType::default());

    init_rustc_env_logger(&early_dcx);
    //signal_handler::install();
    let mut callbacks = AddCallbacks {};
    let using_internal_features = install_ice_hook(DEFAULT_BUG_REPORT_URL, |_| ());
    let exit_code = catch_with_exit_code(|| {
        RunCompiler::new(&args::raw_args(&early_dcx)?, &mut callbacks)
            .set_using_internal_features(using_internal_features)
            .run()
    });

    process::exit(exit_code)
}

Also, my Cargo.toml looks like:

[package]
name = "compiler-plugin-test"
version = "0.1.0"
edition = "2021"

[dependencies]

[package.metadata.rust-analyzer]
rustc_private=true

Meta

rustc --version --verbose:

rustc 1.79.0-nightly (c9f8f3438 2024-03-27)
binary: rustc
commit-hash: c9f8f3438a8134a413aa5d4903e0196e44e37bbc
commit-date: 2024-03-27
host: aarch64-apple-darwin
release: 1.79.0-nightly
LLVM version: 18.1.2

Error output

thread 'rustc' panicked at src/main.rs:49:35:
attempted to read from stolen value: rustc_ast::ast::Crate
Backtrace

stack backtrace:
   0:        0x1011cda20 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h532e10e709153296
   1:        0x101212d50 - core::fmt::write::ha84fc24b2f574968
   2:        0x1011c3c9c - std::io::Write::write_fmt::hc913f40a1f6250b6
   3:        0x1011cd878 - std::sys_common::backtrace::print::h036999af617d91da
   4:        0x1011d01d8 - std::panicking::default_hook::{{closure}}::ha9b16ba0936b1abe
   5:        0x1011cfebc - std::panicking::default_hook::h08b4c81dcc6335ee
   6:        0x10a8135b0 - <alloc[8cd48b23edb49f32]::boxed::Box<rustc_driver_impl[71bebcabe175c2c9]::install_ice_hook::{closure#0}> as core[e2edf039dadc5b61]::ops::function::Fn<(&dyn for<'a, 'b> core[e2edf039dadc5b61]::ops::function::Fn<(&'a core[e2edf039dadc5b61]::panic::panic_info::PanicInfo<'b>,), Output = ()> + core[e2edf039dadc5b61]::marker::Send + core[e2edf039dadc5b61]::marker::Sync, &core[e2edf039dadc5b61]::panic::panic_info::PanicInfo)>>::call
   7:        0x1011d0c1c - std::panicking::rust_panic_with_hook::hcdddd79c60b7b567
   8:        0x1011d05dc - std::panicking::begin_panic_handler::{{closure}}::hf4d2552d8ac43351
   9:        0x1011cdeb0 - std::sys_common::backtrace::__rust_end_short_backtrace::hd305a3c11f56666f
  10:        0x1011d0348 - _rust_begin_unwind
  11:        0x10122c9dc - core::panicking::panic_fmt::hb600c78f45130684
  12:        0x10084b534 - rustc_data_structures::steal::Steal<T>::borrow::h8fb5e50f8322cfc7
                               at /rustc/c9f8f3438a8134a413aa5d4903e0196e44e37bbc/compiler/rustc_data_structures/src/steal.rs:38:13
  13:        0x10083eea8 - <compiler_plugin_test::AddCallbacks as rustc_driver_impl::Callbacks>::after_expansion::haecf194ce1f50cd2
                               at /Users/leo/Projects/compiler-plugin-test/src/main.rs:28:26
  14:        0x10a7d4388 - <rustc_interface[47c26814d8f5e47e]::interface::Compiler>::enter::<rustc_driver_impl[71bebcabe175c2c9]::run_compiler::{closure#0}::{closure#1}, core[e2edf039dadc5b61]::result::Result<core[e2edf039dadc5b61]::option::Option<rustc_interface[47c26814d8f5e47e]::queries::Linker>, rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>
  15:        0x10a805130 - rustc_span[79a02a63db6bf14]::set_source_map::<core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>, rustc_interface[47c26814d8f5e47e]::interface::run_compiler<core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>, rustc_driver_impl[71bebcabe175c2c9]::run_compiler::{closure#0}>::{closure#0}::{closure#0}>
  16:        0x10a8066d0 - rustc_span[79a02a63db6bf14]::create_session_globals_then::<core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>, rustc_interface[47c26814d8f5e47e]::util::run_in_thread_pool_with_globals<rustc_interface[47c26814d8f5e47e]::interface::run_compiler<core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>, rustc_driver_impl[71bebcabe175c2c9]::run_compiler::{closure#0}>::{closure#0}, core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>::{closure#0}>
  17:        0x10a81689c - std[ad43ea9864d59889]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[47c26814d8f5e47e]::util::run_in_thread_with_globals<rustc_interface[47c26814d8f5e47e]::util::run_in_thread_pool_with_globals<rustc_interface[47c26814d8f5e47e]::interface::run_compiler<core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>, rustc_driver_impl[71bebcabe175c2c9]::run_compiler::{closure#0}>::{closure#0}, core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>::{closure#0}, core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>
  18:        0x10a8259a0 - <<std[ad43ea9864d59889]::thread::Builder>::spawn_unchecked_<rustc_interface[47c26814d8f5e47e]::util::run_in_thread_with_globals<rustc_interface[47c26814d8f5e47e]::util::run_in_thread_pool_with_globals<rustc_interface[47c26814d8f5e47e]::interface::run_compiler<core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>, rustc_driver_impl[71bebcabe175c2c9]::run_compiler::{closure#0}>::{closure#0}, core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>::{closure#0}, core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[e2edf039dadc5b61]::result::Result<(), rustc_span[79a02a63db6bf14]::ErrorGuaranteed>>::{closure#1} as core[e2edf039dadc5b61]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  19:        0x1011d9220 - std::sys::pal::unix::thread::Thread::new::thread_start::hbb0f6d2f640f1240
  20:        0x1a7867fa8 - __pthread_joiner_wake
[rustc-ice-2024-03-28T10_08_36-92879.txt](https://github.com/rust-lang/rust/files/14787611/rustc-ice-2024-03-28T10_08_36-92879.txt)

@El-Naizin El-Naizin added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 28, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Mar 28, 2024
@bjorn3
Copy link
Member

bjorn3 commented Mar 28, 2024

Expansion steals the result of the parse query. You either need to access it in the after_crate_root_parsing callback if you want to print the AST before macro expansion (and thus also mod foo; not being expanded into mod foo { ... } yet) or you need to access tcx.resolver_for_lowering().borrow().1 instead if you want to access the post-expansion AST. You can get the tcx using something like queries.global_ctxt().expect("GlobalCtxt stole").borrow_mut().enter(|tcx| { ... }).

@bjorn3 bjorn3 added A-driver Area: rustc_driver that ties everything together into the `rustc` compiler requires-nightly This issue requires a nightly compiler in some way. and removed I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ C-bug Category: This is a bug. labels Mar 28, 2024
@jieyouxu jieyouxu added C-discussion Category: Discussion or questions that doesn't represent real issues. and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Mar 28, 2024
@El-Naizin
Copy link
Author

Oh alright thanks!

In case anyone else had the same question as me, I managed to print out the AST with this code:

#![feature(rustc_private)]

extern crate rustc_ast;
extern crate rustc_data_structures;
extern crate rustc_driver;
extern crate rustc_interface;
extern crate rustc_session;

//use rustc_ast::Crate;
use std::borrow::BorrowMut;
use rustc_data_structures::profiling::get_resident_set_size;
use rustc_driver::{args, catch_with_exit_code, install_ice_hook, DEFAULT_BUG_REPORT_URL};
use rustc_driver::{init_rustc_env_logger, RunCompiler};
use rustc_driver::{Callbacks, Compilation};
use rustc_session::config::ErrorOutputType;
use rustc_session::EarlyDiagCtxt;
use std::process;
use std::time::Instant;

pub struct AddCallbacks {}

impl Callbacks for AddCallbacks {
    fn after_expansion<'tcx>(
        &mut self,
        _compiler: &rustc_interface::interface::Compiler,
        queries: &'tcx rustc_interface::Queries<'tcx>,
    ) -> Compilation {
        queries
            .global_ctxt()
            .expect("Global context stolen")
            .borrow_mut()
            .enter(|tcx| {
                let krate = &tcx.resolver_for_lowering().borrow().1.clone().to_owned();
                println!("Crate found!\n{krate:#?}");
                for item in &krate.items {
                    println!("{item:#?}");
                    if let rustc_ast::ItemKind::Fn(body_id) = &item.kind {
                        println!("Found function!");
                        println!("{body_id:#?}");
                    }
                }
        });
        Compilation::Continue
    }
}

// Main code got from rustc_driver_impl/src/lib.rs/fn main()
// And modified to add custom callbacks
fn main() {
    let _start_time = Instant::now();
    let _start_rss = get_resident_set_size();
    let early_dcx = EarlyDiagCtxt::new(ErrorOutputType::default());

    init_rustc_env_logger(&early_dcx);
    let mut callbacks = AddCallbacks {};
    let using_internal_features = install_ice_hook(DEFAULT_BUG_REPORT_URL, |_| ());
    let exit_code = catch_with_exit_code(|| {
        RunCompiler::new(&args::raw_args(&early_dcx)?, &mut callbacks)
            .set_using_internal_features(using_internal_features)
            .run()
    });

    process::exit(exit_code)
}

(It compiles a rust file, printing out the AST after expansion)

Is it possible to modify the AST now that I accessed it or is it a read-only view?

@bjorn3
Copy link
Member

bjorn3 commented Apr 2, 2024

I think it is technically possible, but I'm not sure if the resolver will like it or if it can cause an ICE or miscompilation due to inconsistencies between it's internal state and the AST. What kind of modification do you want to do?

@El-Naizin
Copy link
Author

The idea was to add code to the start and end of each function (So creating a profile function, and adding a call, to that function at the start and end of every other function).
I wanted to modify the compiler so that I don't have to change the source code with a #[profile] macro, but if it's too difficult to modify the compiler's behaviour, I'm thinking it might be worth it to actually modify the resulting binary.

The benefit from profiling functions with the AST is that cross-compatibility is much easier, and it feels more robust than modifying source code files (I think I can more easily modify the functions of cargo dependencies?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-driver Area: rustc_driver that ties everything together into the `rustc` compiler C-discussion Category: Discussion or questions that doesn't represent real issues. requires-nightly This issue requires a nightly compiler in some way. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

4 participants