RFC: typed-context-injection

[Radbuglet]

Rendered

This RFC proposes the introduction of the Cx structure to the core standard library alongside corresponding compiler support to allow Rust users to conveniently pass "bundles of context" around their applications. Cx is a thin wrapper around tuples of references which adorn the wrapped tuples with the ability to coerce into other Cx objects containing a subset of their references.

[SOF3]

Only skimmed through the rfc quickly.

I understand what motivates the existence of such a type, but what motivates that this type is added to core instead of as a library aside the fact that it cannot be elegantly implemented without compiler magic? Is it more practical to implement language features e.g. generic tuples, where T1 != T2, etc. instead of directly adding a highly specialized library item?

[SOF3]

variadic type parameterized

might as well just assume a tuple and use a syntax similar to Fn?

[Radbuglet]

Yes, it is indeed possible to make Cx take in a single tuple and provide some nice syntactic sugar to transform Cx(&Type1, &Type2) into Cx<(&Type1, &Type2)>. I wrote things this way because I thought it would be easier to implement that way but I am open to changing it if other Rust developers agree that doing so would be more consistent or easier to implement.

[SOF3]

Why must users always use the Cx type? Why wouldn't I define another type with context-like semantics but carry other domain-specific semantics/implications? E.g. I would see this useful in ECS libraries where a function requesting to read certain components causes systems calling this function to also request them.

[Radbuglet]

Users can still use Cx to build their own custom context tuples. For example, an ECS could define its own AccessCx type alias which resolves to a Cx object with appropriately typed access tokens. I don't want to allow users to define their own Cx objects because doing so would prevent useful interoperability between different context types.

With the ECS example, handling AccessCx and regular Cx objects the same way could allow injecting both regular object references and access tokens in the same context like so:

fn my_consumer(cx: Cx<AccessCx<&mut Component1, &Component2>, &MyDependency>) {}

...which would expand to:

fn my_consumer(cx: Cx<Cx<&mut AccessToken<Component1>, &AccessToken<Component2>>, &MyDependency>) {}

[SOF3]

Why do you keep talking of "type aliases"? Type aliases are supposed to be indistinguishable from their values. Meanwhile, here you seem to be mixing up type equality and subtyping.

[SOF3]

Inability to interop between different context types is the whole point why I am mentioning this. It's just like you wouldn't want to transparently pass the resource typemap from rocket to the typemap in serenity just because "they are both contexts".

[Radbuglet]

Inability to interop between different context types is the whole point why I am mentioning this. It's just like you wouldn't want to transparently pass the resource typemap from rocket to the typemap in serenity just because "they are both contexts".

I think part of your misunderstanding of this proposal comes from the fact that Cx is not supposed to be enumerable. Cx refers to an exact type—the tuple of component references you wish to accept. If you don't want a component, you won't get it.

Hence, if Rocket requests a context of components &RocketResource<Thing1> and &RocketResource<Thing2>, and the user has a context Cx<&RocketResource<Thing1>, &RocketUserResource<Thing2>, &RocketResource<Thing3>, &SerenityResource<Thing4>>, it will just receive &RocketResource<Thing1> and &RocketResource<Thing2>—nothing more, nothing less. Hence, it is not dangerous to mix contexts.

Why do you keep talking of "type aliases"? Type aliases are supposed to be indistinguishable from their values. Meanwhile, here you seem to be mixing up type equality and subtyping.

I'm not sure I understand what you mean by this.

[SOF3]

What about Cx<Cx<&System1, &System2>, &System1>? Does it automatically deduplicate to Cx<&System1, &System2>?

[Radbuglet]

If you write out the type:

type MyType<'a> = Cx<Cx<&'a System1, &'a System2>, &'a System1>;

It would indeed be deduplicated to just:

type MyType<'a> = Cx<&'a System1, &'a System2>;

However, if you wrote:

let foo = Cx::new((&system_1, &system_2));
let bar = Cx::new((foo, &system_1));

The types would resolve as follows:

let foo = Cx::new((&system_1, &system_2));  // `foo` has the type `CxRaw<(&System1, &System2)>`.
let bar = Cx::new((foo, &system_1));  // `bar` has the type `CxRaw<(CxRaw<(&System1, &System2)>, &System1)>`.

This process is usually invisible to the user. If they type:

let foo = Cx::new((&system_1, &system_2));
let bar = Cx::new((foo, &system_3));

let baz: Cx<&System1, &System2, &System3> = bar;

The code would be analyzed as follows:

let foo = Cx::new((&system_1, &system_2));  // `foo` has the type `CxRaw<(&System1, &System2)>`.
let bar = Cx::new((foo, &system_3)); // `bar` has the type `CxRaw<(CxRaw<(&System1, &System2)>, &System3)>`.

let baz: Cx<Cx<&System1, &System2>, &System3> = bar;  // This would coerce implicitly to `CxRaw<(&System1, &System2, &System3)>`.

[SOF3]

Cx itself is a type. Why can't I store a context inside a context?

[Radbuglet]

Cx itself is a type. Why can't I store a context inside a context?

You absolutely can store a context in a context—it's just that, when constructing the actual object, it will not be flattened until it needs to.

That... was a bit confusing so let me clarify.

Deduplication and flattening happen at the Cx level and the coercion level, not the CxRaw level. Cx is a type alias for CxRaw which eagerly deduplicates the component list to the best of its ability and only does so once. If that list contains generics, generics will be deduplicated based off their known equalities, not the potential equalities brought on by their substitution by the user. CxRaw is the thing you actually construct when you call Cx::new((..., ...)) and its type is exactly what gets inferred by its constructor. When I say, "it will not be flattened until it needs to," I mean that CxRaw<(CxRaw<&System1, &System2>, &System3)> will only be flattened to a CxRaw<(&System1, &System2, &System3)> during coercion. This isn't something users tend to worry about, however, since this coercion is pretty much invisible.

You might want to reread the section about generics since that explains these rules in much more detail.

[SOF3]

Why coercion instead of explicit conversion e.g. .extract_subset() (or an acronym of it)?

[Radbuglet]

Because methods take ownership of the entire structure while coercion allows us to properly reborrow only the relevant components and construct a new context to be provided.

[SOF3]

That sounds like an argument about having a special syntax to extract a context partially, not blatant coercion. For example,

// cx: Cx<(T, U, V)>
let (t, sub_cx): (&mut T, Cx<(U, V)>) = cx.split::<&mut T>();

[Radbuglet]

Not really. In .splitting a context, you pass the entire context to the function. So, if you have an outstanding borrow on the context, it will be invalidated. In using borrow-aware coercion instead, borrow checking becomes much richer in a way that .split cannot emulate.

fn example(cx: Cx<&mut CommonSystem, &mut System1, &mut System2, &mut OtherSystem>) {
    let borrow_1: Cx<&CommonSystem, &mut System1> = cx;
    let borrow_2: Cx<&CommonSystem, &mut System2> = cx;

    // Both borrows can coexist.
    let _ = (borrow_1, borrow_2);

    // And you can get the values back once you're done!
    let borrow_3: Cx<&mut CommonSystem, &System1> = cx;

    // ...just so long as you don't use `borrow_1` or `borrow_2`.
    // let _ = (borrow_1, borrow_2);
}

And, yeah, you could maybe handle splits manually, but that really isn't convenient when you have a bunch of overlapping contexts that you need to recombine for one function call just to split them again for the next function call.

[SOF3]

This also works with superset contexts right? What about considering making Cx a trait instad so that we do impl<T: Cx<&mut AudioBuffer, &mut VirtualFileSystem, ...>> AudioBufferCxExt for T?

[Radbuglet]

Cx is an object, not a trait. This still works for superset contexts because they implicitly coerce into the appropriate context type.

let foo = Cx::new((&mut vfs, &mut audio_buffer, &mut sound_overrides, &mut audio_cache, &mut other_thing_1, &mut other_thing_2));

// This line...
foo.play_sound_at_path("cbat.ogg");

// Is equivalent to doing...
let foo_borrowed: AudioBufferCx<'_> = foo;  // Uses coercion to get the subset.
foo_borrowed.play_sound_at_path("cbat.ogg");

[SOF3]

My point is exactly saying that Cx should be a trait not a type, because coercing between types arbitrarily like this is... you know, very un-rusty. Very dynamic and confusing to read. imo coercions must not be possible unless they are fully invertible, e.g. Box<T> can be coerced to Box<dyn Trait>, but it is still possible to convert it back to Box<T> if you know the T; converting Cx<(&T, &U)> to &T doesn't make sense because you cannot get back the &U part either way.

[Radbuglet]

I don't really know how to respond to it being "confusing to read." Having used the userland prototype in my own project to inject access tokens, I'd say it certainly helps reduce the messiness of context injection and makes the code easier to read but that's just my opinion.

What I can respond to, however, is the suggestion to make everything a trait. I disagree. In asking for a context inheriting a trait, you say "I want everything in a context that has A, B, and C. Give it all to me!" Compare that to a function taking Cx<&A, &B, &C>, which says "I want the parts of a context with A, B, and C. Do whatever you want with everything else."

Also, in terms of it being "un-rusty," I disagree. We reborrow references with shorter lifetimes all the time. Cx coercion is essentially just an advanced form of reborrowing.

[SOF3]

Lifetime subtyping is transitively equal. If a = b and c = b, a == c is true in a reasonable way. But extracting two different fields from the same context wouldn't make them equal. This completely distorts the meaning of the "equal" symbol. It is very different from reborrowing because it is a completely different type now.

[Radbuglet]

I'm not sure I understand the comment on transitivity. I also don't fully understand the consequences of not upholding some of the properties you list. Does this break the compiler or is this a user expectations problem?

[SOF3]

What if I do this:

fn generic<T: Default, U: Default>() {
    let cx: Cx<T, U> = Cx::new(T::default(), U::default());
    let t: &mut T = cx.extract_mut::<T>();
    let u: &mut U = cx.extract_mut::<U>();
    dbg!(t, u);
}

What happens? There is nothing in the function signature that indicates generic requires T != U. Now we are exposing function implementation to the function signature.

[Radbuglet]

Let us desugar that example.

The example:

fn generic<T: Default, U: Default>() {
    let cx: Cx<T, U> = Cx::new(T::default(), U::default());
    let t: &mut T = cx.extract_mut::<T>();
    let u: &mut U = cx.extract_mut::<U>();
    dbg!(t, u);
}

Desugars to:

fn generic<T: Default, U: Default>() {
    let cx: CxRaw<(T, U)> = CxRaw::new((
        T::default(),
        U::default(),
    ));

    let t: &mut T = &mut *cx.0.0;
    let u: &mut U = &mut *cx.0.1;

    dbg!(t, u);
}

...which works, even if T = V, so there is no need for bounds anywhere!

[SOF3]

Then where is the deduplication part? Now cx contains two values of the same type.

[Radbuglet]

Deduplication only happens once in this entire scheme: in eagerly transforming Cx into CxRaw. This deduplication exists so we can write:

// We just have to change this line from a tuple to a `Cx`.
type Descendant1Cx<'a> = Cx<&'a mut SharedSystem, &'a mut Descendant1System>;

fn descendant_1(cx: Descendant1Cx<'_>) {
    ...
}

// Same here!
type Descendant2Cx<'a> = Cx<&'a mut SharedSystem, &'a mut Descendant2System>;

fn descendant_2(cx: Descendant2Cx<'_>) {
    ...
}

// We also need to give the callee its own context.
type CalleeCx<'a> = Cx<Descendant1Cx<'a>, Descendant2Cx<'a>>;

fn callee(cx: CalleeCx<'_>) {
    descendant_1(cx);
    descendant_2(cx);
}

fn caller(
    shared_system: &mut SharedSystem,
    descendant_1_system: &mut Descendant1System,
    descendant_2_system: &mut Descendant2System,
) {
    callee(Cx::new((
        // ...and now we only have to provide one reference to `shared_system`.
        shared_system,
        descendant_1_system,
        descendant_2_system,
    )));
}

...without issue, even though SharedSystem is, well, shared.

Now cx contains two values of the same type.

That's not a problem since we know exactly where to put these components. T goes to T and V goes to V because that's the only valid choice for all situations.

It becomes a problem, however, when the place from which to take or in which to put the component becomes ambiguous. The canonical case is this:

fn generic_demo<T, V>(cx: Cx<&mut T, &mut V>) {
    let a: &mut T = cx;
    let b: &mut V = cx;
    let _ = (a, b);
}

let mut value = 3u32;
generic_demo::<u32, u32>(Cx::new((&mut value,)));  // Oh no!

It's pretty obvious that, when the user tries to extract T from their context, they want the component corresponding to T and likewise with V.

If those components happen to be the same type, however, and you only have a context with one instance of that type, that's where you have a problem. The code:

let mut value = 3u32;
generic_demo::<u32, u32>(Cx::new((&mut value,)));  // Oh no!

...fails because we have no clue where to put that u32.

Note that this is different from SFINAE. Substitution errors don't snipe the user from deep inside the substitution graph—they snipe them at the call site as if it were any other generic constraint error.

[SOF3]

Why directly = cx instead of = cx.extract() at least?

[Radbuglet]

cx.extract() is implemented in terms of coercion like so:

impl<'a, T: ?Sized> Cx<&'a T> {
    pub fn extract(self) -> &'a T {
        self
    }
}

impl<'a, T: ?Sized> Cx<&'a mut T> {
    pub fn extract_mut(self) -> &'a mut T {
        self
    }
}

So when I do:

let system_1 = cx.extract_mut::<System1>();

That's equivalent to doing:

let cx_subset: Cx<&mut System1> = cx;
let system_1 = cx_subset.extract_mut::<System1>();

[Radbuglet]

Only skimmed through the rfc quickly.

I understand what motivates the existence of such a type, but what motivates that this type is added to core instead of as a library aside the fact that it cannot be elegantly implemented without compiler magic? Is it more practical to implement language features e.g. generic tuples, where T1 != T2, etc. instead of directly adding a highly specialized library item?

I address this in the RFC's alternatives section.

In summary, the contextual injection portion of this proposal can indeed be implemented in userland—here's the playground! Unfortunately, this only supports coercing contexts which are non-nested and without duplicates and I don't think that these features could be implemented in userland. Although the user-land implementation is nonetheless convenient, the lack of these two features means that it still fails to solve the RFC's motivating problem: passing context to deeply nested functions without having to refactor their ancestor functions.

As an example, if you look at this demo of deeply nested context passing:

type Descendant1Cx<'a> = Cx<&'a mut SharedSystem, &'a mut Descendant1System>;

fn descendant_1(cx: Descendant1Cx<'_>) {
    ...
}

type Descendant2Cx<'a> = Cx<&'a mut SharedSystem, &'a mut Descendant2System>;

fn descendant_2(cx: Descendant2Cx<'_>) {
    ...
}

type CalleeCx<'a> = Cx<Descendant1Cx<'a>, Descendant2Cx<'a>>;

fn callee(cx: CalleeCx<'_>) {
    descendant_1(cx);
    descendant_2(cx);
}

fn caller(
    shared_system: &mut SharedSystem,
    descendant_1_system: &mut Descendant1System,
    descendant_2_system: &mut Descendant2System,
) {
    callee(Cx::new((
        // ...and now we only have to provide one reference to `shared_system`.
        shared_system,
        descendant_1_system,
        descendant_2_system,
    )));
}

...we can change the contextual requirements of the descendant functions without updating the callee's context because the callee can merge the two descendant contexts automatically and deduplicate the SharedSystem requirement.

We can't just implement another feature that makes it possible to implement this feature in userland because, in my opinion, the requirements of Cx are too special. The two aforementioned problems can't be solved with better trait reasoning a la specialization, for example, because deduplication is done greedily and will have to deduplicate generic parameters as if they were opaque types rather than wait for them to be concretized.

Unresolved question:

I think there may be a way to implement these two features in userland with type_alias_impl_trait but I don't yet know how I would do it. I'll look into that a bit more and report back if I found a way to implement the feature in userland.

[SOF3]

I think this proposal has a better chance of getting used if we remove all the "context" parts and just focus on the tuple deduplication + extraction logic. For example, maybe a special trait like this is sufficient for extraction (but not deduplication):

pub trait TupleHas<T> {}
// tautologies:
(T, ...): TupleHas<T>
// contradictions:
(U, ...): TupleHas<T> where T != U

And maybe more powerful compiler inference rules to understand what it means by A: NotEquals<B> and its implications.

Then we could just build context tuples on top of these language features, without a highly specific context type.

[matthieu-m]

I agree with @SOF3 here:

1. Tuple manipulation is sufficient for context, and would also work for non-references.
2. Tuple manipulation is more fundamental, and could be used in other contexts (!), so it seems arbitrary to restrict it to just context.

In the end, it seems like a From implementation magically provided by the compiler going from (..., T, ..., U, ..., V, ...) to (U, V, T) is all that would be needed -- in the case where none of the ... provide a T, U, or V at least -- though a different trait may be easier on inference.

[Radbuglet]

I disagree that a trait-based solution would be better. In my opinion, forcing the user to think about negative reasoning only makes their work harder.

Consider, for example, the following rather involved snippet:

// Define "bundle" traits for every type of generic component we're interested in accepting.
trait HasLogger {
    type Logger: Logger;
}

trait HasDatabase {
    type Database: Database;
}

trait HasTracing {
    type Tracing: Tracing;
}

// Define the context for Child1
trait HasBundleForChildCx1: HasLogger + HasDatabase {}

type Child1Cx<'a, B> = Cx<
    &'a mut <B as HasLogger>::Logger,
    &'a mut <B as HasDatabase>::Database,
>;

// Define the context for Child2
trait HasBundleForChildCx2: HasLogger + HasTracing {}

type Child2Cx<'a, B> = Cx<
    &'a mut <B as HasLogger>::Logger,
    &'a mut <B as HasTracing>::Tracing,
>;

// Define the context for parent
type ParentCx<'a, B> = Cx<Child1Cx<'a, B>, Child2Cx<'a, B>>;

trait HasBundleForParent: HasBundleForChildCx1 + HasBundleForChildCx2 {}

// Define our functions
fn parent<B: ?Sized + HasBundleForParent>(cx: ParentCx<'_, B>) {
    let logger: &mut B::Logger = cx;
    let tracing &mut B::Tracing = cx;

    // This already works because of the no-alias assumptions
    // between distinct generic parameters.
    let _ = (logger, tracing);
}

fn caller(cx: Cx<&mut MyLogger, &mut MyDatabase, &mut MyTracing>) {
    consumer::<
        dyn HasBundleForParent<
            Logger = MyLogger,
            Database = MyDatabase,
            Tracing = MyTracing,
        >,
    >(cx);

    // ...which is equivalent to its inferred version:
    consumer::<dyn HasBundleForParent>(cx);
}

Having the user repeat that Logger != Database != Tracing would be super inconvenient, would not automatically scale as we add more components deep down in the context passing hierarchy, and would offer no additional safety to what the RFC already proposes.

Also, such a solution would be drastically more difficult to implement. We'd need negative reasoning and the ability to extract the component from the object using TupleHas without borrowing the entire tuple to even get close to implementing a solution like this.

[SOF3]

nobody is forcing the user to do negative type bounds. this is something to do be done in the library that provides context, which would be implemented the same way if this gets introduced in the standard library anyway.

[Radbuglet]

nobody is forcing the user to do negative type bounds. this is something to do be done in the library that provides context, which would be implemented the same way if this gets introduced in the standard library anyway.

Sorry, I must have misread your original proposal. Let me take a look at it again:

I think this proposal has a better chance of getting used if we remove all the "context" parts and just focus on the tuple deduplication + extraction logic. For example, maybe a special trait like this is sufficient for extraction (but not deduplication):

pub trait TupleHas<T> {}
// tautologies:
(T, ...): TupleHas<T>
// contradictions:
(U, ...): TupleHas<T> where T != U

And maybe more powerful compiler inference rules to understand what it means by A: NotEquals<B> and its implications.

Then we could just build context tuples on top of these language features, without a highly specific context type.

It never crossed my mind that such a solution would be used for type deduplication rather than as the actual coercion mechanism, which does require either partial borrows, a macro a mon userland prototype, or special coercion support. Then, again, I don't really see how that would work here.

I'm a bit confused as to how a trait would handle eager Cx deduplication as described above. How would such a system handle:

fn generic_demo<T, V>(cx: Cx<&mut T, &mut V>) { ... }

Ideally, it should not deduplicate these two parameters ever so that users can write such code without having to specify explicit bounds for the non-overlapping nature of these parameters lest we have the problem I brought up in my last comment.

[Radbuglet]

This proposal seems way too unpopular to ever get accepted so I'm closing this PR to save bandwidth for more promising proposals. Thanks for the feedback, all!

[AnthonyMikh]

I think it is worth mentioning heterogeneous lists from frunk as an alternative. It is not equally ergonomic since the user is forced to mention numerous type bounds, but at least it works today on stable and allows to get elements by type and extract sublists with specified types.