Implement (but don't use) valtree and refactor in preparation of use #82936

oli-obk · 2021-03-09T15:23:00Z

This PR does not cause any functional change. It refactors various things that are needed to make valtrees possible. This refactoring got big enough that I decided I'd want it reviewed as a PR instead of trying to make one huge PR with all the changes.

cc @rust-lang/wg-const-eval on the following commits:

2027184 implement valtree
eeecea9 fallible Scalar -> ScalarInt
042f663 ScalarInt convenience methods

cc @eddyb on ef04a6d

cc @rust-lang/wg-mir-opt for cf1700c (mir::Constant can now represent either a ConstValue or a ty::Const, and it is totally possible to have two different representations for the same value)

rust-highfive · 2021-03-09T15:23:08Z

r? @matthewjasper

(rust-highfive has picked a reviewer for you, use r? to override)

compiler/rustc_middle/src/mir/interpret/value.rs

lcnr · 2021-03-09T20:44:13Z

compiler/rustc_middle/src/mir/mod.rs

@@ -2012,7 +2013,7 @@ impl<'tcx> Operand<'tcx> {
        Operand::Constant(box Constant {
            span,
            user_ty: None,
-            literal: ty::Const::zero_sized(tcx, ty),
+            literal: ConstantSource::Ty(ty::Const::zero_sized(tcx, ty)),


why are you using ConstantSource::Ty instead of Val here?

do you prefer ty::Const values over allocations where possible?

Mostly in order to change as little as possible. ConstantSource::Val is essentially dead code in this PR

compiler/rustc_middle/src/mir/type_foldable.rs

lcnr · 2021-03-09T21:12:48Z

it's kinda unfortunate that ConstantSource has some duplication between it's variants, changing it to something like

enum ConstantSource {
    Param(ParamConst),
    Bound(...),
    Unevaluated(...),
    Value(...),
    Err(...),
}

would be nice, but it's also a bit annoying because we have to somehow change this for the subst folder to work correctly.
using normalize to evaluate the Unevaluated won't work anyways cause that evaluation might not return something representable as a val tree

probably missed/forgot some previous discussion about this

bors · 2021-03-10T04:40:05Z

☔ The latest upstream changes (presumably #82953) made this pull request unmergeable. Please resolve the merge conflicts.

oli-obk · 2021-03-10T11:39:48Z

some duplication between it's variants, changing it to something like

you mean the two occurrences of ConstValue? One of them (the one in ty::Const will become ValTree, but you are right, we will keep having that duplication).

but it's also a bit annoying because we have to somehow change this for the subst folder to work correctly.

Yea, I have a local branch where I tried making ty::Const generic over ValTree or ConstValue to resolve this, but this is way too invasive to do right now. If we do this PR, then we get there in smaller steps. Yes the MIR isn't nice because you have two different representations for the same concrete constant, but the MIR already has that right now (ByRef can technically represent scalars).

The reason this PR is a better first step is that we have to handle all special cases around mir::Constant already without getting the hit on the "make everything generic" solution. Of course we could also "just" have fold_constant in addition to fold_const, but then we'll also get a lot of duplication that I don't know how to handle properly yet (like do we even need Infer, Bound and Param for mir::Constant?). We can explore all these questions with runtime checks before doing the hard thing and cleaning it all up.

lcnr · 2021-03-12T10:57:40Z

would be r=me

@matthewjasper want to still look over this?

valtree is a version of constants that is inherently safe to be used within types. This is in contrast to ty::Const which can have different representations of the same value. These representation differences can show up in hashing or equality comparisons, breaking type equality of otherwise equal types. valtrees do not have this problem.

Value trees won't have scalar ptr at all, so we need a scalar int printing method anyway. This way we'll be able to share that method between all const representations.

This is in preparation of the `literal` field becoming an enum that distinguishes between type level constants and runtime constants

It's not necessary yet, but it may become necessary with things like lazy normalization.

Any use of it has been shown to be a bug in the past.

compiler/rustc_mir/src/const_eval/mod.rs

Co-authored-by: Ralf Jung <post@ralfj.de>

compiler/rustc_middle/src/ty/consts/valtree.rs

Co-authored-by: Ralf Jung <post@ralfj.de>

oli-obk · 2021-03-16T17:57:26Z

@bors r=RalfJung,lcnr,matthewjasper

bors · 2021-03-16T17:57:31Z

📌 Commit c4d564c has been approved by RalfJung,lcnr,matthewjasper

bors · 2021-03-16T22:43:00Z

⌛ Testing commit c4d564c with merge e655fb6...

bors · 2021-03-17T01:38:51Z

☀️ Test successful - checks-actions
Approved by: RalfJung,lcnr,matthewjasper
Pushing e655fb6 to master...

rylev · 2021-03-23T16:07:32Z

@oli-obk @lcnr This seems to have regressed performance slightly in the ctfe stress benchmarks. The regression seems to happen in the eval_to_allocation_raw query. If this change is purely adding unused code, perhaps some inlining has been impacted in the refactoring?

oli-obk · 2021-03-25T15:01:26Z

cachegrind diff:

+1,933,323,286  ???:rustc_mir::interpret::operand::<impl rustc_mir::interpret::eval_context::InterpCx<M>>::const_val_to_op
-1,352,146,900  ???:rustc_mir::interpret::operand::<impl rustc_mir::interpret::eval_context::InterpCx<M>>::const_to_op
+452,987,692  ???:rustc_middle::ty::normalize_erasing_regions::<impl rustc_middle::ty::context::TyCtxt>::subst_and_normalize_erasing_regions
+227,857,068  ???:rustc_mir::interpret::operand::<impl rustc_mir::interpret::eval_context::InterpCx<M>>::eval_operand
+158,899,167  ???:rustc_mir::interpret::step::<impl rustc_mir::interpret::eval_context::InterpCx<M>>::run
+155,194,634  ???:<rustc_middle::ty::layout::LayoutCx<rustc_middle::ty::query::TyCtxtAt> as rustc_target::abi::LayoutOf>::layout_of
+17,034,883  ???:<core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
-14,942,211  ???:<rustc_middle::ty::normalize_erasing_regions::NormalizeAfterErasingRegionsFolder as rustc_middle::ty::fold::TypeFolder>::fold_const
-7,340,200  ???:rustc_middle::mir::interpret::value::ConstValue::try_to_bits
+3,932,250  ???:rustc_middle::mir::interpret::value::ConstValue::try_to_machine_usize

rust-highfive assigned matthewjasper Mar 9, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 9, 2021

This comment has been minimized.

Sign in to view

oli-obk force-pushed the valtree branch from cf1700c to 37f1d90 Compare March 9, 2021 16:05

This comment has been minimized.

Sign in to view

oli-obk force-pushed the valtree branch from 37f1d90 to fd2b8e3 Compare March 9, 2021 17:15

camelid added the A-const-eval Area: constant evaluation (mir interpretation) label Mar 9, 2021

lcnr reviewed Mar 9, 2021

View reviewed changes

oli-obk force-pushed the valtree branch from 18906b6 to 69f1e73 Compare March 10, 2021 12:33

oli-obk added 14 commits March 12, 2021 12:16

Intern valtree field vector

0fe4f38

Add fallible Scalar to ScalarInt conversion method

858216c

Add convenience conversion methods for ScalarInt

09f7f91

Resolve a FIXME around type equality checks in Relate for constants

019dba0

Reduce destructuring and re-interning where possible

4d917fa

Reduce the duplication in the relation logic for constants

5e8a89b

Pull out ConstValue relating into its own function

b729cc9

Split pretty printer logic for scalar int and scalar ptr

0bb367e

Value trees won't have scalar ptr at all, so we need a scalar int printing method anyway. This way we'll be able to share that method between all const representations.

Use the explicit error constant instead of fake dummies

d5eec65

Add ty helper function for mir constants

914df2a

This is in preparation of the `literal` field becoming an enum that distinguishes between type level constants and runtime constants

Directly intern allocations so that we can write a Lift impl for them

3ecde6f

Prepare mir::Constant for ty::Const only supporting valtrees

3127a9c

Replace a custom lift method with a Lift impl

20f7379

oli-obk force-pushed the valtree branch from 69f1e73 to 20f7379 Compare March 12, 2021 12:54

Visit mir::Constant::user_ty for completeness.

6ca1d87

It's not necessary yet, but it may become necessary with things like lazy normalization.

oli-obk added 2 commits March 15, 2021 12:06

Explain pointer and dyn Trait handling in const_to_valtree

0dd5a1b

Do not expose fallible to_int operation on Scalar.

9f407ae

Any use of it has been shown to be a bug in the past.

oli-obk force-pushed the valtree branch from 23bc774 to 9f407ae Compare March 15, 2021 12:07

This comment has been minimized.

Sign in to view

Pacify tidy

1ffd21a

oli-obk mentioned this pull request Mar 16, 2021

normalize mir::Constant differently from ty::Const in preparation for valtrees #83207

Merged

RalfJung reviewed Mar 16, 2021

View reviewed changes

compiler/rustc_mir/src/const_eval/mod.rs Outdated Show resolved Hide resolved

Update compiler/rustc_mir/src/const_eval/mod.rs

f0997fa

Co-authored-by: Ralf Jung <post@ralfj.de>

RalfJung reviewed Mar 16, 2021

View reviewed changes

compiler/rustc_middle/src/ty/consts/valtree.rs Outdated Show resolved Hide resolved

RalfJung reviewed Mar 16, 2021

View reviewed changes

compiler/rustc_middle/src/ty/consts/valtree.rs Outdated Show resolved Hide resolved

oli-obk and others added 2 commits March 16, 2021 18:31

Update compiler/rustc_middle/src/ty/consts/valtree.rs

5b9bd90

Co-authored-by: Ralf Jung <post@ralfj.de>

Update compiler/rustc_middle/src/ty/consts/valtree.rs

c4d564c

Co-authored-by: Ralf Jung <post@ralfj.de>

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 16, 2021

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 17, 2021

bors merged commit e655fb6 into rust-lang:master Mar 17, 2021

rustbot added this to the 1.52.0 milestone Mar 17, 2021

This was referenced Mar 17, 2021

make default anon const substs optional #83086

Closed

extract ConstKind::Unevaluated into a struct #83040

Merged

Add function core::iter::zip #82917

Merged

combine: stop eagerly evaluating consts #81351

Merged

oli-obk deleted the valtree branch March 17, 2021 11:10

Alexendoo mentioned this pull request Mar 18, 2021

broken MIR when transmuting integer to &'static () in const generic #72293

Closed

lcnr mentioned this pull request Dec 11, 2021

Tracking issue for const generics (RFC 2000) #44580

Closed

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement (but don't use) valtree and refactor in preparation of use #82936

Implement (but don't use) valtree and refactor in preparation of use #82936

oli-obk commented Mar 9, 2021 •

edited

rust-highfive commented Mar 9, 2021

This comment has been minimized.

This comment has been minimized.

lcnr Mar 9, 2021

oli-obk Mar 10, 2021

lcnr commented Mar 9, 2021 •

edited

bors commented Mar 10, 2021

oli-obk commented Mar 10, 2021

lcnr commented Mar 12, 2021

This comment has been minimized.

oli-obk commented Mar 16, 2021

bors commented Mar 16, 2021

bors commented Mar 16, 2021

bors commented Mar 17, 2021

rylev commented Mar 23, 2021

oli-obk commented Mar 25, 2021

Implement (but don't use) valtree and refactor in preparation of use #82936

Implement (but don't use) valtree and refactor in preparation of use #82936

Conversation

oli-obk commented Mar 9, 2021 • edited

rust-highfive commented Mar 9, 2021

This comment has been minimized.

This comment has been minimized.

lcnr Mar 9, 2021

Choose a reason for hiding this comment

oli-obk Mar 10, 2021

Choose a reason for hiding this comment

lcnr commented Mar 9, 2021 • edited

bors commented Mar 10, 2021

oli-obk commented Mar 10, 2021

lcnr commented Mar 12, 2021

This comment has been minimized.

oli-obk commented Mar 16, 2021

bors commented Mar 16, 2021

bors commented Mar 16, 2021

bors commented Mar 17, 2021

rylev commented Mar 23, 2021

oli-obk commented Mar 25, 2021

oli-obk commented Mar 9, 2021 •

edited

lcnr commented Mar 9, 2021 •

edited