Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Const generics #2000

Merged
merged 12 commits into from Sep 14, 2017
Merged

Const generics #2000

merged 12 commits into from Sep 14, 2017

Conversation

@withoutboats
Copy link
Contributor

withoutboats commented May 11, 2017

This is a new const generics RFC which addresses the issues around equality between const variables that were raised in this internals thread.

Rendered

[updated to link to final rendered version]

@withoutboats withoutboats self-assigned this May 11, 2017
@withoutboats withoutboats added the T-lang label May 11, 2017
always unknown.

Therefore we can neither unify nor guarantee the nonunification of any const
projection with any other const unless they are *syntactically identical.* That

This comment has been minimized.

@eddyb

eddyb May 11, 2017 Member

My sole nit for this entire RFC is this: "syntactic equality" is, IMO, unactionable.
What I have had in mind is "semantic identity", i.e. what you'd expect from nominal types, where the same node, when used multiple times, unifies with itself.

However, there is another subtlety here: consider unifying two copies of {X / 2}, each with a different inference variable for X. As far as inference is concerned, those variables don't have to be equal. After all, each loses one bit.

cc @nikomatsakis who I believe brought up the same problem with associated types recently.

This comment has been minimized.

@withoutboats

withoutboats May 11, 2017 Author Contributor

Yea I wasn't sure how to phrase this correctly; what I was trying to get across was that {N + 1} should unify with itself in the same way that T::Assoc does.

This comment has been minimized.

@clarfonthey

clarfonthey May 11, 2017 Contributor

I personally would like to avoid the notion of generic equality beyond just X altogether for now. We don't need to add an algebra solver into the compiler, and imo X * 2 and X << 1 and X + X should all be equivalent if we allow this.

This comment has been minimized.

@eddyb

eddyb May 11, 2017 Member

That seems accurate, I wonder how attached @nikomatsakis is to that rule - it's a trade-off.

This comment has been minimized.

@eddyb

eddyb May 11, 2017 Member

@clarcharr There are certainly multiple levels of equivalence we could use.
The worst part IMO is giving more specific results from unification that can be really known.

I do want to eventually treat e.g. {N + 2} and {N + (1 + 1)} as identical, less so have any rules specific to operators or functions, but those future extensions are a bit oit of scope here.

This comment has been minimized.

@withoutboats

withoutboats May 14, 2017 Author Contributor

@eddyb let's talk out of band about what the right wording for this section is. We're starting out more conservatively than I thought (which is fine with me!).

This comment has been minimized.

@gnzlbg

gnzlbg Jun 16, 2017 Contributor

@withoutboats @eddyb I think that saying that "An expression only unifies with itself" and maybe adding @mark-i-m 's example as a clarification (maybe with some comments) would suffice to make it clear what you exactly mean by "with itself".

EDIT: the RFC still needs to be updated with something like this.

This comment has been minimized.

@pnkfelix

pnkfelix Sep 1, 2017 Member

@rfcbot concern expression-unifies-with-itself

Was this thread of discussion ever resolved? On my latest reading (Sept 1), I came away with the impression that two occurrences of the AST {N + 1} (i.e. two different nodes in the AST that both are the subtree {N + 1}) would be considered equal to each other and thus [usize; N + 1] would unify with the type of [0_usize; N + 1].

But @eddyb seems to say that contradicts what he wants to see.

This comment has been minimized.

@pnkfelix

pnkfelix Sep 1, 2017 Member

(or is @eddyb's sole point merely that he anticipates this being an initial implementation limitation, but not a problem with the fundamental design here... I remain confused...)

This comment has been minimized.

@eddyb

eddyb Sep 1, 2017 Member

The initial implementation will consider them distinct, but we can start work on unification strategies after we have anything working at all.

type as itself. (The standard definition of equality for floating point numbers
is not reflexive.)

This may diverse someday from the definition used by match; it is not necessary

This comment has been minimized.

@anp

anp May 11, 2017 Member

nit: diverge?

Because consts must have the structural match property, and this property
cannot be enforced for a type variable, it is not possible to introduce a const
parameter which is ascribed to a type variable (`<T, const N: T>` is not
valid)>

This comment has been minimized.

@hdevalence

hdevalence May 11, 2017

nit: is there an extra > here?

This comment has been minimized.

@withoutboats

withoutboats May 11, 2017 Author Contributor

No, there's just no type name with it, this should be Foo<T, const N: T>.

This comment has been minimized.

@withoutboats

withoutboats May 11, 2017 Author Contributor

oh I saw the wrong >. Yes!

@withoutboats
Copy link
Contributor Author

withoutboats commented May 11, 2017

nits:

  1. I didn't actually talk about the compilation model in this RFC, should probably specify that these params are monomorphized.
  2. Consts as inputs to traits? Unclear to me what the compilation is for that. Monomorphize the traits into distinct traits?
This restriction can be analogized to the restriction on using type variables
in types constructed in the body of functions - all of these declarations,
though private to this item, must be independent of it, and do not have any
of its parameters in scope.

This comment has been minimized.

@Gankra

Gankra May 11, 2017 Contributor

I'm perfectly fine with shipping with this rule, but can you elaborate on... why? It seems unfortunate that this doesn't work:

fn foo<const X: usize>() {
  const STACK_CAP: usize = X * 2;
  let stack1 = ArrayVec<u32; STACK_CAP>::new();
  let stack2 = ArrayVec<u32; STACK_CAP>::new();
  // ...
}

This comment has been minimized.

@withoutboats

withoutboats May 11, 2017 Author Contributor

@gankro The same reason this doesn't work:

fn foo<I: Iterator>(iter: I) {
     fn bar(item: I::Item) { }
}

It would make that internal const a kind of secret associated const of the function, rather than its own item. Obviously this could work someday (even the function example I comment here could work someday) but in the name of incrementalism it's a separate feature.

Possibly we could make an exception for consts (not statics, types, or functions) since they have no representation in the compiled binary. cc @eddyb on this one

This comment has been minimized.

@Gankra

Gankra May 11, 2017 Contributor

Yeah I was only thinking of consts. Since they're basically just named temporaries, it seems totally fine (no weird codegen implications like statics).

This comment has been minimized.

@cramertj

cramertj May 12, 2017 Member

@withoutboats This doesn't work:

fn foo<I: Iterator>(iter: I) -> fn(I::Item) {
     fn bar(item: I::Item) { }
     bar
}
fn bla<I: Iterator>(iter: I) {
    type Bla = I;
}

But this does:

fn foo<I: Iterator>(iter: I) -> fn(I::Item) {
     fn bar<I: Iterator>(item: I::Item) { }
     bar::<I>
}
fn bla<I: Iterator>(iter: I) {
    type Bla<I> = I;
}

So there's already a way to work around it for functions and types. Can you think of a similar way we could make it work for consts and statics? Like @gankro, I think it makes sense for it to "just work" for consts, but I don't know about statics.

This comment has been minimized.

@withoutboats

withoutboats May 12, 2017 Author Contributor

The same problem applies with type arguments in consts and statics today, this doesn't work and there's no way to make it work:

fn foo<I: Iterator>() {
    const NUL: Option<I::Item> = None;
}

I think solving this is the same for both const params and type params, so its an orthogonal RFC from this one.

This comment has been minimized.

@cramertj

cramertj May 12, 2017 Member

there's no way to make it work

To clarify, you mean that there's no way to do this in the language right now, correct?

This comment has been minimized.

@withoutboats

withoutboats May 12, 2017 Author Contributor

Yes! Unlike functions you can't thread a parameter into there. (I think the solution is to make consts Just Work and say sorry about statics).

This comment has been minimized.

@gnzlbg

gnzlbg Jun 16, 2017 Contributor

@withoutboats can you update the RFC with this information? I had exactly this same question.

### Structural equality

Const equality is determined according to the definition of structural equality
defined in [RFC 1445][1445]. Only types which have the "structural match"

This comment has been minimized.

@Gankra

Gankra May 11, 2017 Contributor

typo: "which have" has an extra space in the middle

type as itself. (The standard definition of equality for floating point numbers
is not reflexive.)

This may diverse someday from the definition used by match; it is not necessary

This comment has been minimized.

@Gankra

Gankra May 11, 2017 Contributor

typo: diverse -> diverge

Because consts must have the structural match property, and this property
cannot be enforced for a type variable, it is not possible to introduce a const
parameter which is ascribed to a type variable (`<T, const N: T>` is not
valid)>

This comment has been minimized.

@Gankra

Gankra May 11, 2017 Contributor

Is the structural match property not intended to be exposed as a trait? (why not?)


When comparing the equality of two abstract const expressions (that is, those
that depend on a variable) we cannot compare the equality of their values
because their values are determined by an const variable, the value of which is

This comment has been minimized.

@Gankra

Gankra May 11, 2017 Contributor

typo: an -> a

#### Future extensions

Someday we could introduce knowledge of the basic properties of some operations
- such as the commutitivity of addition and multiplication - to begin making

This comment has been minimized.

@hdevalence

hdevalence May 11, 2017

nit: commutativity


In any sequence of type parameter declarations (such as in the definition of a
type or on the `impl` header of an impl block) const parameters can also be
declared. Const parameters always come after type parameters, and their

This comment has been minimized.

@cramertj

cramertj May 12, 2017 Member

Nit: how does this interact with default type parameters? Can a struct have default type parameters and const parameters? Edit: I ask because default type parameters are required to be listed at the end of the type parameter list.

This comment has been minimized.

@withoutboats

withoutboats May 12, 2017 Author Contributor

I don't feel like I've ever properly understood this concern. Can't we determine from the kind of the node you put there and (usually) how many arguments you've supplied whether it is intended to be a const or a type?

The only case that seems ambiguous to me is something like this:

struct Foo<T = i32, const N: usize = 0>([T; N]);

fn foo<T, const T: usize>(_: Foo<T>) { }

That is you have both const and type default parameters, and you have an ident which is a name in both type and const context (bad news in general), and you supply it once to the type. I don't particularly care what we do here since its such an edge case (probably treat it as the type parameter).

Am I missing something? Why wouldn't this Just Work?

This comment has been minimized.

@cramertj

cramertj May 12, 2017 Member

Oh it very well may "Just Work." I'm just wondering what the plan would be. I think this looks a little odd, for example, since it results in "skipping" a type parameter:

struct Foo<A, B=i32, const N: usize>(A, [B; N]);

fn foo(x: Foo<String, 4>) {...} // The default makes this `Foo<String, i32, 4>`

This comment has been minimized.

@withoutboats

withoutboats May 12, 2017 Author Contributor

I guess I don't think it looks odd because we elide lifetimes all the time (which is problematic, but not in a way that applies here).

This comment has been minimized.

@eddyb

eddyb May 12, 2017 Member

@withoutboats We can only determine whether an identifier is meant to be a type or a constant by checking what its position is declared as - you can right now have both a type and a const defined/imported with the same name in a scope and it disambiguates just fine.

This comment has been minimized.

@gnzlbg

gnzlbg Jun 16, 2017 Contributor

Why is having a type and a const defined/imported with the same name in a scope useful? That is so confusing when talking about const level values that I have to ask whether it wouldn't be better to deprecate it.

This comment has been minimized.

@cramertj

cramertj Sep 5, 2017 Member

I don't think we ever resolved this point. @withoutboats @eddyb Have either of you had any ideas since we discussed? I think it's necessary to have a backwards-compatible way to add default type parameters to things that already have const parameters.

This comment has been minimized.

@withoutboats

withoutboats Sep 5, 2017 Author Contributor

I added an unresolved question about it.

@eddyb
Copy link
Member

eddyb commented May 12, 2017

Consts as inputs to traits? Unclear to me what the compilation is for that. Monomorphize the traits into distinct traits?

What does it mean to "monomorphize" a trait? If const parameters don't match the behavior of type parameters in some context then something has gone wrong with the semantics.

@withoutboats
Copy link
Contributor Author

withoutboats commented May 12, 2017

I'm not sure what I was thinking, its the same as multiparameter traits - we just need to create a new instance during trans for every product of types and consts. That is <T as Foo<0>>::foo generates a different from from <T as Foo<1>>::foo, but this is no different from <T as Foo<Type0>>::foo and <T as Foo<Type1>>::foo.

## When a const variable can be used

A const variable can be used as a const in any of these contexts:

This comment has been minimized.

@matthewjasper

matthewjasper May 12, 2017

In the cases that don't already make this clear, can a const expression involving a const variable also be used in these contexts? And when are they evaluated, e.g. (when) does

impl<const N: usize> SomeType<N> {
     const M: usize = N + usize::MAX
}

error if N > 0?

This comment has been minimized.

@eddyb

eddyb May 12, 2017 Member

I'd say that if it ends up in a type, it can be considered an implement WF requirement for that type, propagating outwards so if it ends up in a concrete type written/inferred, then there would be an error - but if the error comes from monomorphizing a function, it can only be a warning, as per #1229.

@rust-lang/lang might disagree with me, but I think they'd agree we should specify something in this RFC.

This comment has been minimized.

@withoutboats

withoutboats May 12, 2017 Author Contributor

When / why should we do something different than whatever we do when a user writes const M: usze = 1 + usize::MAX;?

This comment has been minimized.

@cramertj

cramertj May 12, 2017 Member

@withoutboats That's an ICE right now on nightly, although it does emit a const-eval error first:

error[E0080]: constant evaluation error
 --> <anon>:4:22
  |
4 |     const M: usize = 1 + ::std::usize::MAX;
  |                      ^^^^^^^^^^^^^^^^^^^^^ attempt to add with overflow

error: internal compiler error: /checkout/src/librustc_trans/mir/constant.rs:377: _1 not initialized
 --> <anon>:5:20
  |
5 |     println!("{}", M);
  |                    ^

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

thread 'rustc' panicked at 'Box<Any>', /checkout/src/librustc_errors/lib.rs:376
note: Run with `RUST_BACKTRACE=1` for a backtrace.

This comment has been minimized.

@withoutboats

withoutboats May 13, 2017 Author Contributor

Unfortunate, but I'm trying to get at what needs to be specified by this RFC (trying to keep it as orthogonal as possible from the const eval system.)

This comment has been minimized.

@mark-i-m

mark-i-m May 13, 2017 Member

It seems like there's const eval and const eval for type unification. The first I agree is orthogonal, but the second I think should be mentioned in the RFC... for example, when are abstract const expressions evaluated (it looks like monomorphization time right now)? do they use the same mechanisms as normal const eval? when are unification errors discovered by the compiler? how does this change the current unification algorithm?

This comment has been minimized.

@eddyb

eddyb May 13, 2017 Member

when are abstract const expressions evaluated

Associated type projections are the analogy here, so: whenever <T as Trait>::Assoc would retry normalizing itself - failure due to dependence on type/const parameters simply results in the projection (abstract expression for constants) not being replaced.

that matching and const parameters use the same definition of equality, but the
definition of equality used by match today is good enough for our purposes.

Because consts must have the structural match property, and this property

This comment has been minimized.

@mark-i-m

mark-i-m May 13, 2017 Member

To be clear, this would work for user-defined types, too, right? As long as they have structural equality? How does this work exactly? Do we just refuse to compile if they use a type that overloads equality? Or is operator overloading irrelevant?

This comment has been minimized.

@withoutboats

withoutboats May 13, 2017 Author Contributor

The RFC for structural_match should answer your questions I think: https://github.com/rust-lang/rfcs/blob/master/text/1445-restrict-constants-in-patterns.md

will be a big project in itself.

However, const generics should be treated as an advanced feature, and it should
not be something we expose to new users early in their use of Rust.

This comment has been minimized.

@mark-i-m

mark-i-m May 13, 2017 Member

So arrays are introduced as magic at first?

This comment has been minimized.

@withoutboats

withoutboats May 13, 2017 Author Contributor

The exact documentation might foremention that you can define your own types with const parameters, but we should avoid bogging users down in a deep understanding of this (or any) feature.

This comment has been minimized.

@gnzlbg

gnzlbg Jun 16, 2017 Contributor

So arrays are introduced as magic at first?

I don't recall what the book says when arrays are introduced, does it say that user defined types can also be parametrized by types? If yes, we should add "and values". Otherwise, I don't see the need.

const X: usize = 7;
let x: RectangularArray<i32, 2, 4>;
let y: RectangularArray<i32, X, {2 * 2}>;

This comment has been minimized.

@petrochenkov

petrochenkov May 13, 2017 Contributor

Minor detail: a resolution ambiguity case is possible:

type X = u8;
const X: u8 = 0;
let _: RectangularArray<i32, X, 0>; // Is `X` a type or a constant?

This needs to be disambiguated in favor of type X for backward compatibility.

(I'm personally mildly against supporting this convenience in the initial implementation, until some experience is gained about how bad RectangularArray<i32, {X}, 0> turns out to be in practice.)

This comment has been minimized.

@eddyb

eddyb May 13, 2017 Member

I believe we can look at the definition to know what to expect from a parameter position.

This comment has been minimized.

@withoutboats

withoutboats May 13, 2017 Author Contributor

Two things:

  1. We could consider disambiguating on the basis of the kinds of the params before falling back to assuming its a type, this is discussed in another comment thread. (Not saying we should, I'm uncertain; there are definitely cons to doing this).
  2. Allowing identity expressions is not intended as a convenience per se but to distinguish them visually from the kinds of const expressions we have to treat as projections. Its primarily pedagogical to help users understand when they can expect certain unification results and when they can't.

This comment has been minimized.

@petrochenkov

petrochenkov May 13, 2017 Contributor

@eddyb

I believe we can look at the definition to know what to expect from a parameter position.

Not in general case.

type X = u8;
const X: u8 = 0;
Type::method<X>; // We can't look at the definition of `method`, it's only available during type checking.
value.method::<X>(); // Same here.

(I don't think just disambiguating in favor of type X will ever cause problems in practice.)

This comment has been minimized.

@withoutboats

withoutboats May 13, 2017 Author Contributor

(I don't think just disambiguating in favor of type X will ever cause problems in practice.)

True!

This comment has been minimized.

@gnzlbg

gnzlbg Jun 16, 2017 Contributor

Having a constant and a type with the same identifier is extremely confusing, and even more so if constants can be "types". Why can't this be deprecated?

This comment has been minimized.

@withoutboats

withoutboats Jun 16, 2017 Author Contributor

I also think it is confusing. Some projects (including the compiler at one point!) take advantage of these two namespaces to create functions with the same names as types to get "constructor syntax." I don't think this is a good idea, and I would be in favor of warning on it, but that's separate from this RFC probably.

This comment has been minimized.

@rkjnsn

rkjnsn Jul 4, 2017 Contributor

Am I remembering correctly that struct Foo; puts Foo in both namespaces?

This comment has been minimized.

@petrochenkov

petrochenkov Jul 4, 2017 Contributor

Am I remembering correctly that struct Foo; puts Foo in both namespaces?

That't true. Also struct Foo(u8, u8);.
So, the namespace separation is used all the time and cannot be deprecated, this is misunderstanding from the previous commenters.

This comment has been minimized.

@rkjnsn

rkjnsn Jul 4, 2017 Contributor

Fortunately, defaulting to the type when there's an ambiguity seems reasonable in both this case and the "constructor syntax" case.

@mark-i-m
Copy link
Member

mark-i-m commented May 13, 2017

Also, where does MIRI fit into this? Is it just that when MIRI comes around consts will suddenly gain a lot of functionality?

Const equality is determined according to the definition of structural equality
defined in [RFC 1445][1445]. Only types which have the "structural match"
property can be used as const parameters. This would exclude floats, for
example.

This comment has been minimized.

@petrochenkov

petrochenkov May 13, 2017 Contributor

Does this include reference types (&T/&mut T)?
They are supported in patterns and use semantic equality (reference targets are compared, not addresses themselves), despite being bitwise comparable ("structural match") as well.

This comment has been minimized.

@petrochenkov

petrochenkov May 13, 2017 Contributor

C++ supports reference const parameters and uses "structural" bitwise comparison to unify them.
http://coliru.stacked-crooked.com/a/3301c82ba77a2f32

This comment has been minimized.

@eddyb

eddyb May 13, 2017 Member

"structural match" for references isn't pointer equality though, they're considered to be equivalent to a newtype for that purpose.

This comment has been minimized.

@withoutboats

withoutboats May 13, 2017 Author Contributor

I believe we should get whatever the match semantics over, so it will compare the targets. Seems very important to be certain we unify two identical string literals even if for whatever reason they are allocated separately in rodata.

@withoutboats
Copy link
Contributor Author

withoutboats commented May 13, 2017

Also, where does MIRI fit into this? Is it just that when MIRI comes around consts will suddenly gain a lot of functionality?

Yes, MIRI is orthogonal. The RFC has a comment in it which says that for the sake of this RFC we just assume integer arithmetic works; we're drawing a distinction between the range of expressions that can be evaluated at compile time (MIRI and const fn) and making types depend on constants (const generics).

values, and cannot implement traits for all arrays.

As a result of this limitation, the standard library only contains trait
implementations for arrays up to a length of 32; as a result, arrays are often

This comment has been minimized.

@est31

est31 May 13, 2017 Contributor

Note: not just the standard library itself has this limitation, but also libraries like serde have it.

@est31
Copy link
Contributor

est31 commented May 13, 2017

👍 🎉 Big support for this RFC and that it tries to not solve every issue related to const generics, but instead goes the slim but faster path, by avoiding to do anything about orthogonal issues.

I too would like to see many of the proposed extensions happen (like unifying {1+N} with {N+1}), but if they were added with this RFC it would increase chance for this RFC to be merged at a later time, and the RFC to hang in unstable limbo for longer, because of some little edge case or because it would require MIRI. I rather have the main functionality merged now, and stabilized fast, and the orthogonal issues looked at separately. Therefore, I'm very glad of the direction this RFC is going!

❤️ Many thanks @withoutboats for championing this RFC!

@clarfonthey
Copy link
Contributor

clarfonthey commented May 13, 2017

Question: does N - 1 unify with usize::MAX?

@mark-i-m
Copy link
Member

mark-i-m commented May 13, 2017

@clarcharr where do you see that?

@withoutboats
Copy link
Contributor Author

withoutboats commented Sep 14, 2017

Const generics is accepted & merged as RFC 2000. Tracking issue is rust-lang/rust#44580.

@@ -12,7 +12,7 @@ allow users to write impls which are abstract over all array types.
# Motivation
[motivation]: #motivation

Rust currently has one type which is parametric over constants: the built-in
Rust currently has one type which is parametric over constants: the built-inf

This comment has been minimized.

@est31

est31 Sep 14, 2017 Contributor

umm...

mqudsi added a commit to mqudsi/rusqlite that referenced this pull request Sep 5, 2018
rust nightly has i128/u128 as 16-byte integer types, expected to land
into stable at some point in the (near?) future. SQLite's maximum
variable-length integer size is 8-bytes, so this needs to be serialized
into a blob for storage in the database.

Obviously endianness is a concern here; I've opted to use the platform's
native endianness rather than a hard-coded BE/LE choice. This is
contracted out to i128's `from_bytes`/`to_bytes`, which will take care
of the platform-specific endianness for us.

The usage of `unsafe` in the call to i128::from_bytes is to avoid a
potentially expensive extra allocation and looping over the contents of
the vector to copy them to a `[u8; 16]` intermediate. Once [RFC #2000](rust-lang/rfcs#2000) lands in nightly (tracked in [#44580](rust-lang/rust#44580)), [const generics](https://internals.rust-lang.org/t/lang-team-minutes-const-generics/5090) should make it possible to convert the `Vec<u8>` to a fixed-length `[u8; 16]` array in a nicer fashion.
@Centril Centril mentioned this pull request Nov 10, 2018
@RustyYato RustyYato mentioned this pull request Jun 8, 2019
@lcnr lcnr mentioned this pull request Jul 28, 2020
2 of 7 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.