Tracking issue for const generics (RFC 2000)

[withoutboats]

Tracking issue for rust-lang/rfcs#2000

cc @eddyb

Blocking stabilization:

• Design:
  • Resolving ordering of const and type parameters, with default parameters
  • Decide what the best UX / implementation cost balance is for unifying abstract const expressions.
  • How we determine well formedness of const expressions.
• Implementation
• Documentation
  • rustc guide

[eddyb]

#44275 added a ConstEvaluatable predicate, and WF([T; expr]) now requires ConstEvaluatable(expr), as expr is lazily evaluated (even if it's just an integer literal). Fulfilling this predicate requires the expression to evaluate successfully, while normalization ignores the error and simply leaves the Unevaluated expression it found, untouched, which is more or less what happens with associated type projections. I expect the same system to scale to const generics.

@EpicatSupercell has expressed interest in working on this, I'll mentor them through the initial implementation. However, we can't go too far because of the limitations described in #44275.

That is, we need @nikomatsakis' lazy normalization to allow constant expressions embedded in types to observe the bounds in scope (from the function / type definition / impl / etc. item they're in), without producing cyclic dependencies half the time.

[eddyb]

Implementation waypoints (for more direct mentoring, seek @eddyb on Gitter or eddyb on IRC):

• Declaration / Syntax:
  • Data structures: AST and HIR - add const parameters alongside type ones
  • Parsing: parse_generic_params - parse const IDENT: Type
  • Misc: visiting (AST, HIR), folding (AST), lowering (HIR), HIR map, DefId collector
• Declaration / Semantics
  • Data structures: ty::Generics - add const parameters alongside type ones
  • Conversion from HIR: generics_of - create ty::ConstParameterDef from hir::ConstParam
• Use / Name Resolution
  • Data structures: Def - add a variant for const parameters
  • Name resolution pass: with_type_parameter_rib - support both type and const generics
• Use / Semantics
  • ConstVal - add Param variant akin to ty::TyParam
  • Conversion from HIR: TyArray with a length that's an ExprPath that resolved to Def::ConstParam should use ConstVal::Param instead of ConstVal::Unevaluated - in a similar fashion to how Def::TyParam turns into ty::TyParam
  • subst::Kind - support &ty::Const and check as_const as well in where places where as_type and as_region are checked
• Inference
  • ConstVal - add InferVar variant akin to ty::ReVar / ty::TyInfer(TyVar(_))
  • InferCtxt - const counterparts to int_unification_table and its UnifyKey impl
  • ty::relate::{Relate,TypeRelation} - support relating ty::Const, and handle ConstVal::InferVar in a similar way to how super_combine_tys does it for types

Note that all of this should allow impl<T, const N: usize> Trait for [T; N] {...}, but not actually passing a constant expression to a type/function, e.g. ArrayVec<T, 3>.

[jplatte]

I'd like to take a stab at this :)

[samsartor]

@jplatte @eddyb Any news on this?

[jplatte]

@samsartor there was a refactoring that had to be done before the main implementation work. That is now almost done (I'm waiting for feedback currently). I don't actually know how much work there is after that. Parsing const params was what I started with initially, before the refactoring. It's not done but I should be able to get that done relatively soon.

[EdorianDark]

@jplatte It would be nice, if you could link the pull request. I was not able to find it.

[jplatte]

There is no PR yet. All my work can be found here.

[RobertWHurst]

Nice work so far @jplatte 🍻

[jplatte]

There is now a PR for the groundwork (Generics refactoring): #45930

[huhlig]

I think this is already included but it would be good to handle the C++ style folding for handling linear algebra style n dimensional arrays with varying lengths, I.E. a 4x4 [[f64;4];4]or a 4x3x5x6 dimensional array [[[[f64;6];5];3];4] and be able to properly wrap and generate specialized methods for both it AND proper trait implementations for scalars properly dimensioned vectors, etc. See https://gist.github.com/huhlig/8b21850b54a75254be4b093551f8c2cb for a rudamentary example.

[Ixrec]

I don't recall anyone proposing fold expressions for Rust before, much less as part of this RFC. But since this is Rust, is there any reason we couldn't implement fold expressions as an ordinary macro, rather than new dedicated syntax?

[yodaldevoid]

This is correct. To give a quick update for those not following the PR #53645, const generics have compile and work in at least one simple usecase. What remains is finishing up codegen for other usecases, including const generics in arrys, and some error output cleanup. After that, the PR should be ready merge and people can start playing with it.

[jeffvandyke]

Might be off-topic, but will this allow a variant or fork of Chunks and related methods to have Item be a compile-time-fixed-size array instead of a slice? This would allow a for loop to destructure/bind to an irrefutable pattern that mapped the elements in the chunk to variables, such as for (first, second) in arr.chunks(2) which currently fails, and I'm guessing (without any justification admittedly) that it would allow more optimization in certain use cases.

[vadixidav]

@jeffvandyke You might be thinking of ChunksExact specifically. Such a change would be API-breaking, so it can't be done. We could add new APIs that give array references in the future, but we can't break existing ones.

[sdroege]

@jeffvandyke You might be thinking of ChunksExact specifically. Such a change would be API-breaking, so it can't be done. We could add new APIs that give array references in the future, but we can't break existing ones.

I'm only waiting for this to be implemented and stabilized before proposing a PR that adds such API in addition to ChunksExact and its variants :)

The runtime and compile-time variants both have their use-cases, you don't always know your chunk size ahead of time. Optimization-wise, if you use ChunksExact with a constant it should be more or less the same according to my testing. The compiler can optimize away all bounds checks.

[shepmaster]

to be implemented and stabilized

I'd suggest not waiting for stabilization as such an API would be one more good use to help exercise this feature for stabilization.

[nhynes]

I'm guessing that this doesn't yet work for impl blocks? I tried

#![feature(const_generics)]

struct The<const Val: u64>();

impl<const Val: u64> The<Val> {
    fn the() {
        println!("{}", Val);
    }
}

but, as was indeed warned, the compiler crashed with

thread 'rustc' panicked at 'slice index starts at 1 but ends at 0', src/libcore/slice/mod.rs:2419:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::_print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: rustc::util::common::panic_hook
   5: std::panicking::rust_panic_with_hook
   6: std::panicking::continue_panic_fmt
   7: rust_begin_unwind
   8: core::panicking::panic_fmt
   9: core::slice::slice_index_order_fail
  10: rustc_resolve::Resolver::resolve_ident_in_lexical_scope
  11: rustc_resolve::Resolver::resolve_path
  12: rustc_resolve::Resolver::resolve_path_without_parent_scope
  13: rustc_resolve::Resolver::smart_resolve_path_fragment
  14: rustc_resolve::Resolver::smart_resolve_path
  15: <rustc_resolve::Resolver<'a> as syntax::visit::Visitor<'tcx>>::visit_ty
  16: syntax::visit::walk_generic_args
  17: syntax::visit::walk_ty
  18: rustc_resolve::Resolver::with_generic_param_rib
  19: rustc_resolve::Resolver::resolve_item
  20: rustc_resolve::Resolver::resolve_crate
  21: rustc::util::common::time
  22: rustc_interface::passes::configure_and_expand_inner
  23: rustc_interface::passes::configure_and_expand::{{closure}}
  24: <rustc_data_structures::box_region::PinnedGenerator<I, A, R>>::new
  25: rustc_interface::passes::configure_and_expand
  26: <rustc_interface::queries::Query<T>>::compute
  27: <rustc_interface::queries::Query<T>>::compute
  28: <rustc_interface::queries::Query<T>>::compute
  29: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::prepare_outputs
  30: rustc_interface::interface::run_compiler_in_existing_thread_pool
  31: <std::thread::local::LocalKey<T>>::with
  32: <scoped_tls::ScopedKey<T>>::set
  33: syntax::with_globals

The error is related to the const in impl<const Val: u64> since removing that part causes other errors but doesn't crash.

but props to Rust for even considering this feature. I had no idea if it'd work but the syntax seemed natural, I went for it, and lo rustc said it existed :)

[HadrienG2]

I'm not surprised that it is not working, as this hotly anticipated feature is still being plumbed through the compiler as we speak (see e.g. #59008 and #58581 and the earlier work on #53645 which was abandoned because the PR was too big, but still kept open as a tracker to announce progress).

However, I'm not sure if out-of-bounds slice accesses should be expected of the current implementation stubs. @varkor @yodaldevoid, can you have a look?

[varkor]

Yes, the warning is correct: const generics are not yet functional in any form. There are still a few more pull requests before they're ready to start getting played around with.

[Coder-256]

Sorry if this is not the right place to ask questions but I couldn't find anywhere better. Just 2 questions:

1. Can a single function be conditionally const? For example, a function could have 2 signatures like:

   const fn foo<A: const T>(x: T)  // `A` const implements `T`
   fn foo<A: T>(x: A)              // `A` implements `T` normally

   In this case, foo is const iff A: const T; if A does not const implement T, it still satisfies the bound but foo is no longer const. It is also important for the author to be able to specify any generic bound for more complex examples (ex. where A::Output : Bar). A great example of this is even simple arithmetic:

   // This only accepts types that const implement `T`
   const fn square_const<T: const Mul>(x: T) -> T {
     x*x
   }
   
   // This accepts types that implement `T` any way, but it is not const
   // This function behaves identically to `square_const`
   // But has a different signature and needs a different name
   fn square<T: Mul>(x: T) -> T {
     square_const(x)
   }
   
   let a: u8 = 5;
   let b: FooNumber = FooNumber::new();
   square_const(a); // `u8` const implements Mul
   square(b); // `FooNumber` implements `Mul` normally, so we need a separate function?

   I feel strongly that there should definitely be a way to do this, and I'm surprised that it is not mentioned in the RFC (unless I missed it?).

2. [less important]: Will there be a way to detect in the body of a const function whether we are running at compile-time or run-time? I think that some macro similar to cfg!() would be a useful extension, if for no other reason than debugging. cfg!() is currently evaluated at compile-time already, so I think(/guess) that it should be able to know whether or not the function is being used as const or a regular function since that too is determined at compile-time. However this is less important than my 1st question.

[rpjohnst]

@Coder-256

1. Yes, see rust-lang/rfcs#2632.
2. I am not sure whether that should even be possible, though given the above I'm not sure it's necessary either.

[yodaldevoid]

@Coder-256 Both of these questions are not related to const generics, but rather const functions. Const generics are for being generic over consts (e.g. foo<2>()) rather than functions being able to be run at compile-time. I imagine this is why you did not find the answers to your questions in RFC 2000.

[Coder-256]

@rpjohnst Thank you, but I think I might have been unclear. I have already seen both rust-lang/rfcs#2632 and rust-lang/rfcs#2000, but I'm pretty sure that this is not mentioned in either. (but I could be wrong?) What I am asking about is conditionally const functions. See the examples I wrote since it is hard to describe.

@yodaldevoid Whoops you're right, where should I ask this?

As for the macro question, I agree that there isn't really much use for it now that I think about it

[varkor]

What I am asking about is conditionally const functions. See the examples I wrote since it is hard to describe.

Your square_const definition can be used in place of square (that is: it is coerced to a function with the equivalent signature at run-time). See rust-lang/rfcs#2632 for discussion of this behaviour (also, that thread is the right place to ask questions about any interactions between const fn and trait bounds).

[Coder-256]

@varkor I'm not convinced that that's the case (since the trait bounds change), but I'll ask in rust-lang/rfcs#2632.

[Jezza]

Crash Report:

Code:

#![feature(const_generics)]

use std::marker::PhantomData;

struct BoundedU32<const LOWER: u32, const UPPER: u32> {
    value: u32,
    _marker: PhantomData<(LOWER, UPPER)>,
}

Compiler:

thread 'rustc' panicked at 'slice index starts at 1 but ends at 0', src/libcore/slice/mod.rs:2419:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.35.0-nightly (719b0d984 2019-03-13) running on x86_64-unknown-linux-gnu

note: compiler flags: -C codegen-units=1 -C debuginfo=2 --crate-type lib

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `playground`.

To learn more, run the command again with --verbose.

[varkor]

@Jezza: const generics isn't fully implemented yet and is not expected to work. We'll make an announcement when it's time to start experimenting with #![feature(const_generics)] (which you'll be notified about if you're subscribed to this issue).

[eddyb]

@varkor Hang on, @Jezza's example has _marker: PhantomData<(LOWER, UPPER)> - those are two const parameters used as types, why didn't rustc_resolve produce an error?

[varkor]

Good point: I'll investigate this. Note that this is only a problem with #![feature(const_generics)], so it's not a critical issue (using non-generic consts produces the error as expected). Maybe it never reaches rustc_resolve.

[varkor]

@eddyb: this ICE is coming from resolve_ident_in_lexical_scope, so I imagine it's probably related to #58307.

Edit: actually, maybe not — that seems to apply just to macro_rules!.

[petrochenkov]

Minimized:

#![feature(const_generics)]

struct S<const C: u8>(C);

Resolve ICEs before producing the "expected type, found value" error.

The index is out-of-bounds here:
https://github.com/rust-lang/rust/blob/master/src/librustc_resolve/lib.rs#L3919