Composable StackState

[alexito4]

Hello,

Recently we started investigating what would take to start using the new NavigationStack and related TCA APIs. During this investigation we’ve encountered limitations of the tools when it comes to having navigation in a modularized application where each feature has its own module and they are composed together in an "integrator" app.

We've been working on a prototype that aims to solve these limitations and would like to share it with the community and get a conversation started to see if the solution makes sense (or we are just going in the wrong direction) and if there is any chance the navigation tools in the TCA could evolve to provide a similar solution.

We're open to any feedback, on the general concept and specific solution. And if there is any misconception or misunderstanding we've made during the explanation by all means shout ahead ^^

Video demoing the results. Integrator is the main app target, Feature A and B are different SPM packages, each providing its own navigation logic that is composed together in the integrator.

Simulator.Screen.Recording.-.iPhone.15.-.2024-05-16.at.17.59.04.mp4

What follows is a detail explanation of our exploration. You can see the prototype in this repo.

The limitations

The documentation does a great job of explaining how to use NavigationSack in the TCA and what are the tradeoffs of StackState vs NavigationPath. But some of those tradeoffs are big enough that it has prompted a few questions and conversations in GH discussions and the Slack community about a better way of composing navigation stacks across modularized features.

Just so we're on the same page, this is how to use NavigationStack and StackState with a modularized app:

1. Each Feature provides the screen reducers (with state and action) and views.
2. An integrator (usually the app target) provides the NavigationStackView
3. The integrator implements an enum Path reducer that has cases for the reducers of all features screens.
4. The integrator implements a Reducer that reacts to the actions sent in the Features and has the navigation logic that manipulates the StackState (push, pop...)

This structure has the main problem of forcing the navigation logic to be implemented in the integrator instead of letting each feature module have its own logic and let the integrator just compose it together. So far in our 4 year journey with the TCA we’ve been able to encapsulate all internal logic of a Feature on its own module, just calling out via delegate actions to the integrator for navigating to other features and letting the integrator compose the feature as its pleases. This ease for composition is what we love the most about the TCA and what sets it a part from other architectures. But with the navigation tools we feel like we're forced to diverge from this mindset. For modularised apps this is problematic, not only because the feature no longer encapsulates its own navigation, but also because the navigation logic would have to be implemented on every "integrator" app.

Reviewing the points from above to see what it's already good and what we wish we could improve:

1. Number 1 is fine, a feature provides the screens to the integrator. ✅
2. Number 2 is fine, ✅ (although in an ideal scenario NavigationStackView would be composable on its own, so a feature could provide its own and the integrator could just nest it on another one. But SwiftUI doesn’t like this so we need to accept there is only 1 stack view in the hierarchy)
3. Number 3 is fine too, the integrator is the one composing the features. ✅
4. Number 4 is the problematic, we want the navigation logic on each feature. ⚠️

With this analysis in mind we tried to focus on fixing the last point. It was important to us to keep as close to the current TCA tools and APIs as possible, reducing the code we have to maintain or the changes that would theoretically be needed on the TCA to offer such tools. In other words, we didn’t want to reinvent huge new approaches, we just wanted to find the path of least resistance to compose navigation in a satisfying way.

What should a feature see?

As soon as you start trying to modularise the navigation you quickly realise that the types will fight you. The StackState wants a type that enums all features, but your feature doesn’t know about those, so you can’t simply pass the stack as is to each feature.

As a side note, moving the types to its own module is not a feasible solution in our opinion, as that brings its own set of problems and sort of breaks the modularizion we have in place. Features shouldn't know about other features.

So then, the general idea is to provide a stack to each feature that only has its own Path type. But the question is, what should that stack contain? In general there would be two approaches:

A feature stack only sees the screens from the feature itself

This means that each feature would have its own array (StackState) and somehow the integrator would merge those arrays into a true stack state that drives the NavigationStackView.

This brings a bunch of problems and questions that are hard to answer. Like, what happens when the flow goes to feature A, then to B, then to A again, for example. And the logic to do this "merge" is not straightforward either.

But most important than the complexities of the code is the conceptual reasoning behind this approach, which we consider very weird. If you think about it, It means the code in a feature thinks is alone on the stack when in reality is sharing the navigation stack it with other features, or even other instances of itself.

We consider features to be independent and isolated, yes, but there is always an implicit assumption that the feature is part of some bigger app. That’s already part of the architecture and we shouldn’t shy away from it. We just consider the limitations and assumptions made.

A feature stack contains all screens, but the feature can only access the types of its own

This is the approach we preferred and took. This means the stack a feature gets is the same as the integrator has, so it contains all path states for all the screens in the stack, from your feature and others. This means it can get real counts and in general manipulate the stack array as usual.

The only difference is how the types of the elements are exposed, as the feature can only see the ones it owns. How we solve this is the crux of the problem. We also liked that this is how most developers are used to see the navigation stack, as for example in UIKit you have access to it and see its entirety all the time.

The explored solution

To accomplish this solution we basically have to "type erase" the types of the elements that are not part of the feature.

As a side note, our first approach that didn’t require any changes or wrappers was to make the feature Reducer that contains the StackState generic. That meant the feature didn’t know which type it contained, and passing the StackState from the integrator to the feature meant the type was sort of "erased" automatically by the compiler automatically.

At this point no matter which way we go it seems that to erase the StackState element type we need to give to the feature a sort of "view" of the array. To do this we opted for mapping the StackState to a new element type, giving it to the Feature, and then mapping it back to the original StackState.

StackState mapping

The blocker we encountered is that StackState protects its internal data which is not visible to the outside. Creating a new StackState means new IDs and generations getting out of sync breaking a bunch of things. (that said even with the IDs broken some stuff worked fine which proved to us the potential of the solution)

What we did to solve this was modify the StackState a bit to open a way for us to keep the same IDs but changing the element type to another one. See the commit diff here martin-muller@43f5c7e
We understand this might not be desirable, and it needs polish and expert review if it ever was to get in the lib, but it was the key to unlock us. There might also be other ways of type erasing that we haven’t been able to make work.

Our hope is that even if the overall solution is not good enough for the community, at least we can find a way to open StackState a bit for us to maintain the rest of the code (I really don't want to fork the TCA). Or if there are other ways that don't need StackState to be opened, even better!

Which type the Feature gets

So with the feature being able to receive a StackState of a different type than the integrator while keeping both in sync, the next question is what is that type.

We started with Any, as is the most erased type we could have. And that worked quite well, with this we could have the feature modularized and composed in the integrator. ❤️ We could have an extension in StackState that brought some type safety back, internally checking if the type was the one your feature expected, but that is just nice sugar. The system works fine with it.

The problem with Any is that there is no safety at all. Of course we want to erase some type information, is the whole point of the exercise, but we still want to do it in a safe way. Otherwise one could add types that the integrator doesn't know how to deal with (you could append an Int instead of a proper reduce state!).

To recover some safety we introduced a StackElement enum with two cases, a typed one for the feature internal screens, and an untyped one for the external screens of other features. This means the feature is able to access all elements, and for the ones that it owns is able to access them fully typed. For the external ones it still can access them but it can't do much with them since what it gets is a fully opaque type. We also added a couple of convenience methods to make the general use cases (like appending your types the feature owns) work directly, so most of the time StackElement fades into the background.

The shape of the Feature

With this tools in place it means we can implement a Feature by:

1. provide its exposed screen reducers as a Path enum reducer as we're used to. See code.
2. Implementing a Reducer that contains its navigation logic, what we cal the StackReducer of the feature

As you can see the code looks just like normal TCA StackState code, just with a wrapper in the element type, and keeping the navigation logic inside the feature.

The integrator

Then the last piece of puzzle was to put together everything in the integrator. We have a high order Reducer (https://github.com/martin-muller/navigation-play/blob/main/TCAExtensions/Sources/TCAExtensions/StackReducer.swift#L94) that is capable of doing the type erasure of the StackState for a specific feature. We see this as a special version of a Scope.

And at this point we have what we wanted! You can just use Scope and StackReducer in the integrator reducer and compose all feature navigation together. 👌

But we did an extra step just to try to have a nicer API inline with the great ones that come from the TCA. For that we made a forEach overload that lets you add the StackReducer's as part of the standard for each.

        .forEach(\.path, action: \.path) {
            StackReducer(state: \.featureA, action: \.featureA) {
                FeatureA()
            }
            
            StackReducer(state: \.featureB, action: \.featureB) {
                FeatureB()
            }
        }

And we’re exploring if there is some magic, maybe with macros, that can just write that automatically so we can just keep the existing .forEach() API. The tricky part is that the feature "Path" enum and the reducer that has the navigation logic "navigator" are separated entities, so we would need a way to bring them together, maybe with a protocol. But anyway, that’s for sugar that is not needed.

Conclusion

The prototype seems to have worked and covers our needs very well and we’ve tried to do it with the minimum amount of changes possible. We realise the StackState change might be a contention point, and that the StackElement adds another type, but we consider the tradeoffs worth it.

You can see working in the repo https://github.com/martin-muller/navigation-play/ with a prototype that has 2 features and 1 integrator. It’s a simple example but one that already shows real world navigation.

We hope this helps as a start of a conversation and we would be happy to get pointed to different solutions if they exist. But we do think some solution is needed to fill the lack of composability in navigation logic.

Thanks to @martin-muller for the great work on this. And to you for reading this ^^