Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICE while compiling tower for tonic on nightly #101844

Closed
grtlr opened this issue Sep 15, 2022 · 9 comments · Fixed by #101902
Closed

ICE while compiling tower for tonic on nightly #101844

grtlr opened this issue Sep 15, 2022 · 9 comments · Fixed by #101902
Labels
C-bug Category: This is a bug. E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ P-critical Critical priority T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@grtlr
Copy link

grtlr commented Sep 15, 2022

An ICE occurs when compiling tower as dependency for tonic:0.8.1 using the rustc nightly compiler.

Code

main.rs:

fn main() {
    println!("Hello World");
}

Cargo.toml:

[package]
name = "ice-tower"
version = "0.1.0"
edition = "2021"

[dependencies]
tonic = "=0.8.1"

I've created a repository with a Cargo.lock file to help reproduce this error:
https://github.com/grtlr/rustc-ice-tower-2022-09-15

Please let me know if there is anything that I can do to help solve this. Thank you!

Meta

rustc --version --verbose:

rustc 1.65.0-nightly (750bd1a7f 2022-09-14)
binary: rustc
commit-hash: 750bd1a7ff3e010611b97ee75d30b7cbf5f3a03c
commit-date: 2022-09-14
host: x86_64-apple-darwin
release: 1.65.0-nightly
LLVM version: 15.0.0

Error output

RUST_BACKTRACE=full cargo +nightly build                                                                                                                                                                                                                          (main|✔) ~/D/ICE-tower 
   Compiling tower v0.4.13
   Compiling tower-http v0.3.4
error: internal compiler error: no errors encountered even though `delay_span_bug` issued

error: internal compiler error: broken MIR in DefId(0:345 ~ tower[ab58]::balance::pool::{impl#3}::build) (CanonicalUserTypeAnnotation { user_ty: Canonical { max_universe: U0, variables: [CanonicalVarInfo { kind: Ty(General(U0)) }, CanonicalVarInfo { kind: Ty(General(U0)) }, CanonicalVarInfo { kind: Ty(General(U0)) }, CanonicalVarInfo { kind: Ty(General(U0)) }], value: TypeOf(DefId(0:232 ~ tower[ab58]::balance::p2c::service::{impl#1}::new), UserSubsts { substs: [^0, ^1], user_self_ty: Some(UserSelfTy { impl_def_id: DefId(0:229 ~ tower[ab58]::balance::p2c::service::{impl#1}), self_ty: balance::p2c::service::Balance<^2, ^3> }) }) }, span: /Users/goertler/.cargo/registry/src/github.com-1ecc6299db9ec823/tower-0.4.13/src/balance/pool/mod.rs:293:22: 293:34 (#0), inferred_ty: fn(std::pin::Pin<std::boxed::Box<balance::pool::PoolDiscoverer<MS, Target, Request>>>) -> balance::p2c::service::Balance<std::pin::Pin<std::boxed::Box<balance::pool::PoolDiscoverer<MS, Target, Request>>>, Request> {balance::p2c::service::Balance::<std::pin::Pin<std::boxed::Box<balance::pool::PoolDiscoverer<MS, Target, Request>>>, Request>::new} }): bad user type AscribeUserType(fn(std::pin::Pin<std::boxed::Box<balance::pool::PoolDiscoverer<MS, Target, Request>>>) -> balance::p2c::service::Balance<std::pin::Pin<std::boxed::Box<balance::pool::PoolDiscoverer<MS, Target, Request>>>, Request> {balance::p2c::service::Balance::<std::pin::Pin<std::boxed::Box<balance::pool::PoolDiscoverer<MS, Target, Request>>>, Request>::new}, DefId(0:232 ~ tower[ab58]::balance::p2c::service::{impl#1}::new) UserSubsts { substs: [_, _], user_self_ty: Some(UserSelfTy { impl_def_id: DefId(0:229 ~ tower[ab58]::balance::p2c::service::{impl#1}), self_ty: balance::p2c::service::Balance<_, _> }) }, type_of=fn(D) -> balance::p2c::service::Balance<D, Req> {balance::p2c::service::Balance::<D, Req>::new}): NoSolution
  |
  = note: delayed at compiler/rustc_borrowck/src/type_check/mod.rs:1087:25

thread 'rustc' panicked at 'Box<dyn Any>', compiler/rustc_errors/src/lib.rs:1492:13
stack backtrace:
  <see below>

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.65.0-nightly (750bd1a7f 2022-09-14) running on x86_64-apple-darwin

note: compiler flags: --crate-type lib -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack
error: could not compile `tower`
warning: build failed, waiting for other jobs to finish...
Backtrace

stack backtrace:
   0:        0x109646552 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h4c5ecb0a45d5e886
   1:        0x10969dcca - core::fmt::write::h1b48d1d1cb0b7961
   2:        0x1096386dc - std::io::Write::write_fmt::hd88196c3a88e34fb
   3:        0x10964943b - std::panicking::default_hook::{{closure}}::h0f6810d654ce88bd
   4:        0x109649147 - std::panicking::default_hook::h99ae6a40b3c0b158
   5:        0x1120bbcca - rustc_driver[a68fcf032531d799]::DEFAULT_HOOK::{closure#0}::{closure#0}
   6:        0x109649b6e - std::panicking::rust_panic_with_hook::h6b1861a02d4bcbed
   7:        0x116aa9a87 - std[67b2ec307cdf3306]::panicking::begin_panic::<rustc_errors[503eaa35d8a7773]::ExplicitBug>::{closure#0}
   8:        0x116aa9219 - std[67b2ec307cdf3306]::sys_common::backtrace::__rust_end_short_backtrace::<std[67b2ec307cdf3306]::panicking::begin_panic<rustc_errors[503eaa35d8a7773]::ExplicitBug>::{closure#0}, !>
   9:        0x116e0ddf9 - std[67b2ec307cdf3306]::panicking::begin_panic::<rustc_errors[503eaa35d8a7773]::ExplicitBug>
  10:        0x116a9eaa9 - std[67b2ec307cdf3306]::panic::panic_any::<rustc_errors[503eaa35d8a7773]::ExplicitBug>
  11:        0x116aa354e - <rustc_errors[503eaa35d8a7773]::HandlerInner as core[cfcc7bd811e8a7b7]::ops::drop::Drop>::drop
  12:        0x1120eb0ba - core[cfcc7bd811e8a7b7]::ptr::drop_in_place::<rustc_session[157c61238769214e]::parse::ParseSess>
  13:        0x1120f1afb - <alloc[e8b3202084de779]::rc::Rc<rustc_session[157c61238769214e]::session::Session> as core[cfcc7bd811e8a7b7]::ops::drop::Drop>::drop
  14:        0x1120d6db2 - core[cfcc7bd811e8a7b7]::ptr::drop_in_place::<rustc_interface[fcdf76b1f16a6137]::interface::Compiler>
  15:        0x1120dbe9f - rustc_interface[fcdf76b1f16a6137]::interface::create_compiler_and_run::<core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>, rustc_driver[a68fcf032531d799]::run_compiler::{closure#1}>
  16:        0x112133895 - <scoped_tls[fdde64bf4d39370e]::ScopedKey<rustc_span[a6701603fc349d9]::SessionGlobals>>::set::<rustc_interface[fcdf76b1f16a6137]::interface::run_compiler<core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>, rustc_driver[a68fcf032531d799]::run_compiler::{closure#1}>::{closure#0}, core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>>
  17:        0x1120f8ada - std[67b2ec307cdf3306]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[fcdf76b1f16a6137]::util::run_in_thread_pool_with_globals<rustc_interface[fcdf76b1f16a6137]::interface::run_compiler<core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>, rustc_driver[a68fcf032531d799]::run_compiler::{closure#1}>::{closure#0}, core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>>::{closure#0}, core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>>
  18:        0x1120fc1f8 - <<std[67b2ec307cdf3306]::thread::Builder>::spawn_unchecked_<rustc_interface[fcdf76b1f16a6137]::util::run_in_thread_pool_with_globals<rustc_interface[fcdf76b1f16a6137]::interface::run_compiler<core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>, rustc_driver[a68fcf032531d799]::run_compiler::{closure#1}>::{closure#0}, core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>>::{closure#0}, core[cfcc7bd811e8a7b7]::result::Result<(), rustc_errors[503eaa35d8a7773]::ErrorGuaranteed>>::{closure#1} as core[cfcc7bd811e8a7b7]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  19:        0x109652ad7 - std::sys::unix::thread::Thread::new::thread_start::hf1224c03c7001e01
  20:     0x7ff80bf854e1 - __pthread_start

note: the compiler unexpectedly panicked. this is a bug.

@grtlr grtlr added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Sep 15, 2022
@grtlr grtlr changed the title ICE while compiling tower for tonic ICE while compiling tower for tonic on nightly Sep 15, 2022
@Alex6323
Copy link

Same here.

rustc 1.65.0-nightly (750bd1a7f 2022-09-14)
binary: rustc
commit-hash: 750bd1a7ff3e010611b97ee75d30b7cbf5f3a03c
commit-date: 2022-09-14
host: x86_64-unknown-linux-gnu
release: 1.65.0-nightly
LLVM version: 15.0.0

@nmoutschen
Copy link

Same here, managed to isolate it to tower with the balance feature. See this repo: https://github.com/nmoutschen/repro-ice/blob/main/Cargo.toml#L9

rustc 1.65.0-nightly (750bd1a7f 2022-09-14)
binary: rustc
commit-hash: 750bd1a7ff3e010611b97ee75d30b7cbf5f3a03c
commit-date: 2022-09-14
host: aarch64-apple-darwin
release: 1.65.0-nightly
LLVM version: 15.0.0

@jackh726
Copy link
Member

This seems to be getting hit quite a bit. Let's figure out which PR is responsible for this.

@jackh726 jackh726 added the E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc label Sep 15, 2022
@lqd
Copy link
Member

lqd commented Sep 15, 2022

I believe it's a926696 and of these, #101433 looks like the most likely.

update: #101433's perf artifacts, b512bbede20f6f1ec7a77e2a226dbc38578f118a, do reproduce the ICE, so I'm pretty sure it's that one.

@jackh726 jackh726 added P-critical Critical priority and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Sep 15, 2022
@jackh726
Copy link
Member

I'm actually just going to mark as critical, since this has been hit by multiple people and beta cut is tomorrow

@jackh726
Copy link
Member

Gah okay. Well, I guess let's work on getting an MCVE. I'm not sure if I'll have time to investigate a proper fix, but I can at least put up a revert.

@jackh726 jackh726 added the E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example label Sep 15, 2022
@lqd lqd removed the E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc label Sep 15, 2022
@Nilstrieb
Copy link
Member

I am trying to minimize this in https://github.com/Nilstrieb/101844-repro right now

@Nilstrieb
Copy link
Member

Nilstrieb commented Sep 15, 2022

I've minimized it down to 70 freestanding lines of complex trait impls. I don't have time to minimize it further, so if anyone wants to look into it more, feel free :)
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=f657596f7a4b63350356399bd9ec40da

@teor2345
Copy link
Contributor

Same here, also due to tonic:

$ rustc -vV
rustc 1.65.0-nightly (cf9ed0dd5 2022-09-15)
binary: rustc
commit-hash: cf9ed0dd5836201843d28bbad50abfbe1913af2a
commit-date: 2022-09-15
host: x86_64-unknown-linux-gnu
release: 1.65.0-nightly
LLVM version: 15.0.0

jackh726 added a commit to jackh726/rust that referenced this issue Sep 16, 2022
o0Ignition0o pushed a commit to apollographql/router that referenced this issue Sep 16, 2022
> **Note**
> We're almost to 1.0! We've got a couple relatively small breaking
changes to the configuration for this release (none to the API) that
should be relatively easy to adapt to and a number of bug fixes and
usability improvements.

## ❗ BREAKING ❗

### Change `headers` propagation configuration ([PR
#1795](#1795))

While it wasn't necessary today, we want to avoid a necessary breaking
change in the future by proactively making room for up-and-coming work.
We've therefore introduced another level into the `headers`
configuration with a `request` object, to allow for a `response` (see
[Issue #1284](#1284)) to
be an _additive_ feature after 1.0.

A rough look at this should just be a matter of adding in `request` and
indenting everything that was inside it:

```patch
headers:
    all:
+     request:
          - remove:
              named: "test"
```

The good news is that we'll have `response` in the future! For a full
set of examples, please see the [header propagation
documentation](https://www.apollographql.com/docs/router/configuration/header-propagation/).

By @bnjjj in #1795

### Bind the Sandbox on the same endpoint as the Supergraph, again
([Issue #1785](#1785))

We have rolled back an addition that we released in this week's
`v1.0.0-rc.0` which allowed Sandbox (an HTML page that makes requests to
the `supergraph` endpoint) to be on a custom socket. In retrospect, we
believe it was premature to make this change without considering the
broader impact of this change which ultimately touches on CORS and some
developer experiences bits. Practically speaking, we may not want to
introduce this because it complicates the model in a number of ways.

For the foreseeable future, Sandbox will continue to be on the same
listener address as the `supergraph` listener.

It's unlikely anyone has really leaned into this much already, but if
you've already re-configured `sandbox` or `homepage` to be on a custom
`listen`-er and/or `path` in `1.0.0-rc.0`, here is a diff of what you
should remove:

```diff
sandbox:
-  listen: 127.0.0.1:4000
-  path: /
  enabled: false
homepage:
-  listen: 127.0.0.1:4000
-  path: /
  enabled: true
```

Note this means you can either enable the `homepage`, or the `sandbox`,
but not both.

By @o0Ignition0o in #1796

## 🚀 Features

### Automatically check "Return Query Plans from Router" checkbox in
Sandbox ([Issue
#1803](#1803))

When loading Sandbox, we now automatically configure it to toggle the
"Request query plans from Router" checkbox to the enabled position which
requests query plans from the Apollo Router when executing operations.
These query plans are displayed in the Sandbox interface and can be seen
by selecting "Query Plan Preview" from the drop-down above the panel on
the right side of the interface.

By @abernix in #1804

## 🐛 Fixes

### Fix `--dev` mode when no configuration file is specified ([Issue
#1801](#1801)) ([Issue
#1802](#1802))

We've reconciled an issue where the `--dev` mode flag was being ignored
when running the router without a configuration file. (While many use
cases do require a configuration file, the Router actually doesn't
_need_ a confguration in many cases!)

By @bnjjj in #1808

### Respect `supergraph`'s `path` for Kubernetes deployment probes
([Issue #1787](#1787))

If you've configured the `supergraph`'s `path` property using the Helm
chart, the liveness
and readiness probes now utilize these correctly. This fixes a bug where
they continued to use the _default_ path of `/` and resulted in a
startup failure.

By @damienpontifex in #1788

### Get variable default values from the query for query plan condition
nodes ([PR #1640](#1640))

The query plan condition nodes, generated by the `if` argument of the
`@defer` directive, were
not using the default value of the variable passed in as an argument.

This _also_ fixes _default value_ validations for non-`@defer`'d
queries.

By @Geal in #1640

### Correctly hot-reload when changing the `supergraph`'s `listen`
socket ([Issue
#1814](#1814))

If you change the `supergraph`'s `listen` socket while in `--hot-reload`
mode, the Router will now correctly pickup the change and bind to the
new socket.

By @o0Ignition0o in #1815

## 🛠 Maintenance

### Improve error message when querying non existent field [Issue
#1816](#1816)

When querying a non-existent field you will get a better error message:

```patch
{
  "errors": [
    {
-       "message": "invalid type error, expected another type than 'Named type Computer'"
+       "message": "Cannot query field \"xxx\" on type \"Computer\""
    }
  ]
}
```

By @bnjjj in #1817

### Update `apollo-router-scaffold` to use the published `apollo-router`
crate [PR #1782](#1782)

Now that `apollo-router` is released on
[crates.io](https://crates.io/crates/apollo-router), we have updated the
project scaffold to rely on the published crate instead of Git tags.

By @o0Ignition0o in #1782

### Refactor `Configuration` validation [Issue
#1791](#1791)

Instantiating `Configuration`s is now fallible, because it will run
consistency checks on top of the already run structure checks.

By @o0Ignition0o in #1794

### Refactor response-formatting tests
[#1798](#1798)

Rewrite the response-formatting tests to use a builder pattern instead
of macros and move the tests to a separate file.

By @Geal in #1798

## 📚 Documentation

### Add `rustdoc` documentation to various modules ([Issue
#799](#799))

Adds documentation for:

- `apollo-router/src/layers/instrument.rs`
- `apollo-router/src/layers/map_first_graphql_response.rs`
- `apollo-router/src/layers/map_future_with_request_data.rs`
- `apollo-router/src/layers/sync_checkpoint.rs`
- `apollo-router/src/plugin/serde.rs`
- `apollo-router/src/tracer.rs`

By @garypen in #1792

### Fixed `docs.rs` publishing error from our last release

During our last release we discovered for the first time that our
documentation wasn't able to compile on the [docs.rs](https://docs.rs)
website, leaving our documentation in a [failed
state](https://docs.rs/crate/apollo-router/1.0.0-rc.0/builds/629200).

While we've reconciled _that particular problem_, we're now being
affected by
[this](https://docs.rs/crate/router-bridge/0.1.7/builds/629895) internal
compiler errors (ICE) that [is
affecting](rust-lang/rust#101844) anyone using
`1.65.0-nightly` builds circa today. Since docs.rs uses `nightly` for
all builds, this means it'll be a few more days before we're published
there.

With thanks to @SimonSapin in
apollographql/federation-rs#185

Co-authored-by: Coenen Benjamin <benjamin.coenen@hotmail.com>
@bors bors closed this as completed in 4d4e51e Sep 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ P-critical Critical priority T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants