Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly stall coroutine witnesses in new solver #138845

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

compiler-errors
Copy link
Member

TODO: write description

r? lcnr

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver) labels Mar 22, 2025
@rustbot
Copy link
Collaborator

rustbot commented Mar 22, 2025

Some changes occurred to the core trait solver

cc @rust-lang/initiative-trait-system-refactor

changes to inspect_obligations.rs

cc @compiler-errors, @lcnr

@@ -65,13 +65,13 @@ pub enum TypingMode<I: Interner> {
/// let x: <() as Assoc>::Output = true;
/// }
/// ```
Analysis { defining_opaque_types: I::DefiningOpaqueTypes },
Analysis { defining_opaque_types: I::LocalDefIds, stalled_generators: I::LocalDefIds },
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably could squash this into one list.

@@ -162,7 +162,7 @@ where
self.select(selcx)
}

fn drain_unstalled_obligations(
fn drain_stall_obligations_for_coroutines(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fn drain_stall_obligations_for_coroutines(
fn drain_stalled_obligations_for_coroutines(

ScrubbedTraitError<'tcx>,
>(self.at, ct, vec![None; ct.outer_exclusive_binder().as_usize()])
{
Ok((value, _)) => value,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We throw away ambiguous preds here b/c we may be using this folder on types that really are just not fully normalizable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The preds list should just contain stalled coroutine obligations, after all.

/// entered before passing `value` to the function. This is currently needed for
/// `normalize_erasing_regions`, which skips binders as it walks through a type.
///
/// TODO: doc
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to explain that this doesn't return all ambiguous preds, just the ones that are stalled on coroutines.

struct StalledOnCoroutines<'tcx> {
stalled_generators: &'tcx ty::List<LocalDefId>,
span: Span,
// TODO: Cache
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache would be nice since visiting everything 128318913 times to look for coroutines is probably expensive.

@@ -189,6 +189,20 @@ where
debug_assert!(ecx.opaque_type_is_rigid(opaque_ty.def_id));
}

if let ty::CoroutineWitness(def_id, _) = goal.predicate.self_ty().kind() {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should pull this out into a helper, b/c I think I need to also apply this hack to copy/clone. I think those are it tho.

// Increase this limit if necessary, but do try to keep the size low if possible
#[cfg(target_pointer_width = "64")]
const _: () = {
if size_of::<Key<'static>>() > 88 {
if size_of::<Key<'static>>() > 96 {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:((

@rust-log-analyzer

This comment has been minimized.

jhpratt added a commit to jhpratt/rust that referenced this pull request Mar 24, 2025
Tweaks to writeback and `Obligation -> Goal` conversion

Each of these commits are self-contained, but are prerequisites that I'd like to land before rust-lang#138845, which still needs some cleaning.

The ""most controversial"" one is probably [Explicitly don't fold coroutine obligations in writeback](rust-lang@e7d27ba), which I prefer because I think using `fold_predicate` to control against not normalizing predicates seems... easy to mess up 🤔, and we could have *other things* that we don't want to normalize.

Explicitly noting whether we want `resolve` to normalize is a lot clearer (and currently in writeback is limited to resolving stalled coroutine obligations), since we can attach it to a comment that explains *why*.
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 24, 2025
Rollup merge of rust-lang#138846 - compiler-errors:stall-prereqs, r=lcnr

Tweaks to writeback and `Obligation -> Goal` conversion

Each of these commits are self-contained, but are prerequisites that I'd like to land before rust-lang#138845, which still needs some cleaning.

The ""most controversial"" one is probably [Explicitly don't fold coroutine obligations in writeback](rust-lang@e7d27ba), which I prefer because I think using `fold_predicate` to control against not normalizing predicates seems... easy to mess up 🤔, and we could have *other things* that we don't want to normalize.

Explicitly noting whether we want `resolve` to normalize is a lot clearer (and currently in writeback is limited to resolving stalled coroutine obligations), since we can attach it to a comment that explains *why*.
@bors
Copy link
Contributor

bors commented Mar 24, 2025

☔ The latest upstream changes (presumably #138873) made this pull request unmergeable. Please resolve the merge conflicts.

}

fn pending_obligations(&self) -> PredicateObligations<'tcx> {
self.obligations.clone_pending()
}

fn drain_unstalled_obligations(&mut self, _: &InferCtxt<'tcx>) -> PredicateObligations<'tcx> {
self.obligations.take_pending()
fn drain_stall_obligations_for_coroutines(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fn drain_stall_obligations_for_coroutines(
fn drain_stalled_obligations_for_coroutines(

fn visit_goal(&mut self, inspect_goal: &super::inspect::InspectGoal<'_, 'tcx>) -> Self::Result {
inspect_goal.goal().predicate.visit_with(self)?;

if let Some(candidate) = inspect_goal.unique_applicable_candidate() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this type visitor feels somewhat fragile and I expect unique_applicable_candidate and the limited recursion depth to cause us to fail to stall obligations in very rare cases. otoh I don't think this is a problem though

so my understanding here is:

  • for correctness it doesn't matter how many obligations we stall
  • for diagnostics (and perf) we want to stall as few obligations as possible
  • failing to stall causes unexpected ambiguity errors

Please add this as a comment somewhere, prolly the stalled_coroutine_obligations field of the typeck results

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that's my understanding. We could perhaps stall obligations if we find coroutines in the predicate or if we hit the recursion limit, but idk if we have a facility to detect when we hit the recursion limit here. Shouldn't be too hard to fix, but I'd rather leave that to when we need it.

@compiler-errors
Copy link
Member Author

Let's see how bad the perf is from making items larger.

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 25, 2025
@compiler-errors
Copy link
Member Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 25, 2025
…<try>

Properly stall coroutine witnesses in new solver

TODO: write description

r? lcnr
@bors
Copy link
Contributor

bors commented Mar 25, 2025

⌛ Trying commit b6f1961 with merge 5443aaa4127ecdfcad1a50e7d7f2e4650bb52877...

@rust-log-analyzer
Copy link
Collaborator

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
info: removing rustup binaries
info: rustup is uninstalled
##[group]Image checksum input
mingw-check-tidy
# We use the ghcr base image because ghcr doesn't have a rate limit
# and the mingw-check-tidy job doesn't cache docker images in CI.
FROM ghcr.io/rust-lang/ubuntu:22.04

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
  g++ \
  make \
---

COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---
#12 2.881 Building wheels for collected packages: reuse
#12 2.882   Building wheel for reuse (pyproject.toml): started
#12 3.104   Building wheel for reuse (pyproject.toml): finished with status 'done'
#12 3.105   Created wheel for reuse: filename=reuse-4.0.3-cp310-cp310-manylinux_2_35_x86_64.whl size=132720 sha256=0c2fd2aaf7b0bf8d6e131220aff14712a774c2ca462f3204d25460cbcf610b63
#12 3.106   Stored in directory: /tmp/pip-ephem-wheel-cache-o8mw1w4n/wheels/3d/8d/0a/e0fc6aba4494b28a967ab5eaf951c121d9c677958714e34532
#12 3.108 Successfully built reuse
#12 3.108 Installing collected packages: boolean-py, binaryornot, tomlkit, reuse, python-debian, markupsafe, license-expression, jinja2, chardet, attrs
#12 3.526 Successfully installed attrs-23.2.0 binaryornot-0.4.4 boolean-py-4.0 chardet-5.2.0 jinja2-3.1.4 license-expression-30.3.0 markupsafe-2.1.5 python-debian-0.1.49 reuse-4.0.3 tomlkit-0.13.0
#12 3.526 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#12 4.075 Collecting virtualenv
#12 4.113   Downloading virtualenv-20.29.3-py3-none-any.whl (4.3 MB)
#12 4.178      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.3/4.3 MB 69.1 MB/s eta 0:00:00
#12 4.238 Collecting filelock<4,>=3.12.2
#12 4.247   Downloading filelock-3.18.0-py3-none-any.whl (16 kB)
#12 4.282 Collecting platformdirs<5,>=3.9.1
#12 4.286   Downloading platformdirs-4.3.7-py3-none-any.whl (18 kB)
#12 4.305 Collecting distlib<1,>=0.3.7
#12 4.309   Downloading distlib-0.3.9-py2.py3-none-any.whl (468 kB)
#12 4.318      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 469.0/469.0 KB 67.2 MB/s eta 0:00:00
#12 4.401 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#12 4.589 Successfully installed distlib-0.3.9 filelock-3.18.0 platformdirs-4.3.7 virtualenv-20.29.3
#12 4.590 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#12 DONE 4.7s

#13 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#13 DONE 0.0s
---
DirectMap4k:      128960 kB
DirectMap2M:     7211008 kB
DirectMap1G:    11534336 kB
##[endgroup]
Executing TIDY_PRINT_DIFF=1 python2.7 ../x.py test            --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
+ TIDY_PRINT_DIFF=1 python2.7 ../x.py test --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
##[group]Building bootstrap
    Finished `dev` profile [unoptimized] target(s) in 0.05s
##[endgroup]
WARN: currently no CI rustc builds have rustc debug assertions enabled. Please either set `rust.debug-assertions` to `false` if you want to use download CI rustc or set `rust.download-rustc` to `false`.
[TIMING] core::build_steps::tool::LibcxxVersionTool { target: x86_64-unknown-linux-gnu } -- 0.228
---
fmt check
fmt: checked 5927 files
tidy check
tidy: Skipping binary file check, read-only filesystem
##[error]tidy error: /checkout/compiler/rustc_ty_utils/src/opaque_types.rs:355: TODO is used for tasks that should be done before merging a PR; If you want to leave a message in the codebase use FIXME
##[error]tidy error: /checkout/compiler/rustc_trait_selection/src/solve/fulfill.rs:249: TODO is used for tasks that should be done before merging a PR; If you want to leave a message in the codebase use FIXME
##[error]tidy error: /checkout/compiler/rustc_trait_selection/src/solve/normalize.rs:63: TODO is used for tasks that should be done before merging a PR; If you want to leave a message in the codebase use FIXME
removing old virtual environment
creating virtual environment at '/checkout/obj/build/venv' using 'python3.10' and 'venv'
creating virtual environment at '/checkout/obj/build/venv' using 'python3.10' and 'virtualenv'
Requirement already satisfied: pip in ./build/venv/lib/python3.10/site-packages (25.0.1)
linting python files
All checks passed!
checking python file formatting
26 files already formatted
checking C++ file formatting
some tidy checks failed
Command has failed. Rerun with -v to see more details.
Build completed unsuccessfully in 0:01:50
  local time: Tue Mar 25 16:58:20 UTC 2025
  network time: Tue, 25 Mar 2025 16:58:20 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

@bors
Copy link
Contributor

bors commented Mar 25, 2025

☀️ Try build successful - checks-actions
Build commit: 5443aaa (5443aaa4127ecdfcad1a50e7d7f2e4650bb52877)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (5443aaa): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

mean range count
Regressions ❌
(primary)
0.3% [0.1%, 0.5%] 71
Regressions ❌
(secondary)
0.3% [0.1%, 0.5%] 38
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 0.3% [0.1%, 0.5%] 71

Max RSS (memory usage)

Results (primary 1.2%, secondary -1.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.5% [0.5%, 3.9%] 18
Regressions ❌
(secondary)
2.5% [1.0%, 3.9%] 3
Improvements ✅
(primary)
-1.6% [-2.5%, -0.7%] 2
Improvements ✅
(secondary)
-3.7% [-6.8%, -0.9%] 7
All ❌✅ (primary) 1.2% [-2.5%, 3.9%] 20

Cycles

Results (secondary -1.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
2.0% [2.0%, 2.0%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.7% [-4.5%, -1.0%] 2
All ❌✅ (primary) - - 0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 777.999s -> 780.062s (0.27%)
Artifact size: 365.81 MiB -> 365.88 MiB (0.02%)

@rustbot rustbot added the perf-regression Performance regression. label Mar 25, 2025
@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Mar 25, 2025
@compiler-errors
Copy link
Member Author

Let me try putting coroutines into the same list as the opaques 🤔

@lcnr
Copy link
Contributor

lcnr commented Mar 25, 2025

alternatively, intern TypingEnv itself. We should only very rarely access its value and it's already 2 ptrs wide

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf-regression Performance regression. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants