RFC: Allow layout router to retain state when navigating within dynamic segments

[jkuhs]

Goal

Allow client-side navigation within the same dynamic segment without all components being unmounted/remounted.

The Issue

Currently, when navigating client-side from one path to another within the same dynamic route segment, e.g. from /en to /de, the layout router will cause all components on the page to get unmounted and remounted.

More specifically, the layout router renders <TemplateContext.Provider key={createRouterCacheKey(preservedSegment, true)}> whereas the cache key created by createRouterCacheKey includes the dynamic values when within a dynamic segment.

Consider the following example:

Configured Route	URL Path	Route Segment Value	Cache Key
/app/[lang]/page.js	/en	['lang', 'en', 'd']	lang|en|d
/app/[lang]/page.js	/de	['lang', 'de', 'd']	lang|de|d

With the above example, navigating client-side from /en to /de will cause all components on the page to get unmounted and remounted, even though we remain in the same dynamic segment ([lang]) and on the same page. This is due to the cache key including the dynamic value, i.e. changing from lang|en|d to lang|de|d.

The same is applies for any other files, such as layout.js, that might be located in a dynamic segment.

This is problematic for various reasons, including:

1. Degraded performance - Unmounting and remounting whole component trees when only the context (i.e. accessible via useParams, useSearchParams) has changed, is wasteful and unnecessary.
2. Loss of tree-bound state - It's common to bind and maintain state in the component tree (e.g. using useState and useRef). Oftentimes, that state is used for context providers that wrap the application near the "root" level. Remounting the whole component tree results in all that state being lost and being re-initialized (or even failing).
3. Undesired side effects - Any useEffect's defined by remounted components will run again, possibly causing undesired side effects and unnecessary resources to be used (e.g. redundant requests, repeated init logic, etc.).
4. Inconsistent - This issue happens on dynamic path parameter changes but not on search parameter changes, even if in both cases the tree structure/components remain the same. It's also inconsistent with how the page router works.
5. Issues with UI transitions and UI states - Remounting the whole tree can prevent certain UI transitions (e.g. animations) from working. It can also lead to certain UI states to be reset (e.g. nested scroll container) when they should retain their current state.

Background

The page router's _app.jsx behaviour is to keep state between navigations and to instead manually pass <Component key={path} {...pageProps} /> to force a state reset on navigation.

Personally, I think the page router's approach is correct and it should be up to developers to force a state reset when and where they need it, rather than Next.js forcing it.

Proposal

Update the logic in the layout router and the createRouterCacheKey function to omit any dynamic values when using the cache key as key property on <TemplateContext.Provider>. Effectively, that will make it work the same as it already does for search parameters.

Using the initial example from above, the created cache key would be lang|d for both the /en and the /de routes (instead of lang|en|d and lang|de|d respectively).

Updating the logic could simply be implemented with a change along the lines of (simplified):

// layout-router.tsx
<TemplateContext.Provider key={createRouterCacheKey(preservedSegment, { withoutDynamicValues: true })} ...>

// create-router-cache-key.ts
export function createRouterCacheKey(
  segment: Segment,
  options?: {
    withoutDynamicValues?: boolean
  }
) {
  if (options?.withoutDynamicValues) {
    if (Array.isArray(segment)) {
      return `${segment[0]}|${segment[2]}`
    }
    return segment.startsWith('__PAGE__') ? '__PAGE__' : segment
  }
  
  return Array.isArray(segment)
    ? `${segment[0]}|${segment[1]}|${segment[2]}`
    : segment
}

If there are concerns about omitting dynamic values of dynamic segments by default, then a compromise would be to add a configuration option for developers to opt-in to the behavior. For example:

// next.config.js
module.exports = {
  experimental: {
    appDir: true,
    disableDynamicSegmentRemount: true
  },
}

Examples of various routes, their cache keys without dynamic values, and the resulting React tree

The purpose of the following examples is to demonstrate that omitting the dynamic values is OK and doesn't cause issues due to conflicting cache keys.

Keep in mind, that we are only omitting the dynamic values when creating the cache key for the key property on <TemplateContext.Provider>. Cache keys created for other purposes (e.g. route specific data caching) will still include the dynamic values!

Note, that the React trees below are simplified and only show the relevant <TemplateContext.Provider> components with their key properties. There are other components and properties rendered in the layout router that are not shown here since they are not relevant to illustrate the issue at hand.

Static page with search parameters

This already works today. The dynamic part (?{"my-search-param":"true"}) is omitted.

Route:	/app/blog/page.js
URL:	/blog?my-search-param=true
Segment value:	__PAGE__?{"my-search-param":"true"}
Cache key:	__PAGE__
React tree:	<TemplateContext.Provider key="blog"> <TemplateContext.Provider key="__PAGE__">

Root dynamic segment

Route:	/app/[path]/page.js
URL:	/blog
Segment value:	['path', 'blog', 'd']
Cache key:	path|d
React tree:	<TemplateContext.Provider key="path|d"> <TemplateContext.Provider key="__PAGE__">

Root dynamic catch-all segment

Route:	/app/[...path]/page.js
URL:	/blog	/blog/one
Segment value:	['path', 'blog', 'c']	['path', 'blog/one', 'c']
Cache key:	path|c
React tree:	<TemplateContext.Provider key="path|c"> <TemplateContext.Provider key="__PAGE__">

Root dynamic optional catch-all segment

Route:	/app/[[...path]]/page.js
URL:	/blog	/blog/one
Segment value:	['path', 'blog', 'oc']	['path', 'blog/one', 'oc']
Cache key:	path|oc
React tree:	<TemplateContext.Provider key="path|oc"> <TemplateContext.Provider key="__PAGE__">

Nested dynamic segment

Route:	/app/blog/[path]/page.js
URL:	/blog/one
Segment value:	blog for the static `blog` directory segment ['path', 'one', 'd'] for the dynamic `[path]` segment
Cache key:	path|d
React tree:	<TemplateContext.Provider key="blog"> <TemplateContext.Provider key="path|d"> <TemplateContext.Provider key="__PAGE__">

[jkuhs]

@timneutkens I have made the necessary changes on my local (canary branch) following the contributor guidelines. I implemented it so that it only applies when the configuration experimental.disableDynamicSegmentRemount is set to true, so I don't think there is any risk in adding it. Happy to create a PR for you to review.

[danvoyce]

I've just hit this same issue and have seen a few other related (and closed) issues.

I've tried to use route groups but it doesn't work for my scenario.

@timneutkens I appreciate you guys have a lot going on, but this does seem like a very common problem / regression compared to the pages router, so wondering why this RFC is has been neglected.

I've created an MRE https://codesandbox.io/p/devbox/currying-dust-jf5mtr

Please help 🙏

[MateoKappa]

Hello, i am also having the same problem ! i am having a hard time understanding how i should implement this is my layout , can you give a littile more detailed example ?

[jkuhs]

@MateoKappa The above is just my proposal for addressing the issue. Hopefully, someone from the Next.js team will respond and accept the RFC for implementation. At that point, I would be happy to create a PR to get the necessary changes merged into a future Next.js version.

In the meantime, I have come with this simple patch that you can run to fix the issue. Note, that it modifies the Next.js build in your node_modules directory.

patch-next.js

const fs = require('fs');

const filePath = './node_modules/next/dist/client/components/router-reducer/create-router-cache-key.js';

try {
  const dataBuffer = fs.readFileSync(filePath);
  const updatedData = dataBuffer
    .toString()
    .replace(
      'Array.isArray(segment) ? segment[0] + "|" + segment[1] + "|" + segment[2]',
      'Array.isArray(segment) ? withoutSearchParameters ? segment[0] + "|" + segment[2] : segment[0] + "|" + segment[1] + "|" + segment[2]'
    );
  fs.writeFileSync(filePath, updatedData);
  console.log('Successfully patched create-router-cache-key.js');
} catch (err) {
  console.log('Could not patch create-router-cache-key.js', err);
}

Simply run this in your project directory via command: node patch-next.js

[MateoKappa]

Alright thank you !

[danvoyce]

Gave this a go @jkuhs with no avail 😕

[jkuhs]

@danvoyce It has been a while since I wrote that patch and there might have been changes to the Next.js source code that make it not work anymore. We have given up on Next.js for now.

[chirayu-miles]

This bit us while trying to upgrade one of our internal apps as a PoC to the app router. It also affects our other internals apps.

Here's a simplified example. We have a section of our app that is fully dynamic.

/app/kpi/[...slug]/layout.tsx
/app/kpi/[...slug]/page.tsx

/app/kpi/[...slug]/layout.tsx render a navigation drawer on the left hand side containing a tree of links. This tree is large and is scrollable. Our users scroll this navigation drawer to an area of their interest, and visit a few links by clicking one after another. With the app router, each time they click a link, the navigation drawer scrolls back all the way to the top because the component is destroyed and remounted—the dynamic segments are in the key of the TemplateContext.Provider. This is not an issue with the pages router (existing app.) We can't work around this by rendering the drawer in the page.tsx (without a layout) as the TemplateContext.Provider wraps them both and they're both remounted.

Hoping to see a solution to this.

ps: Hoping that layout.tsx (and page.tsx) can preserve state for those using layout given that anyone who wants the unmount/mount behavior can simply use template.tsx as per the documentation.

[gnoff]

Hi @chirayu-miles

Is there a reason this is not a suitable composition?

/app/kpi/layout.tsx
/app/kpi/[...slug]/page.tsx

In this case the layout is above the dynamic route and will not remount when transitioning between different slugs

If you can't lift the layout to the kpi folder because there are non catchall routes at that level that shouldn't share the layout then you can put it into a route group

/app/kpi/(kpi)/layout.tsx -> contains nav drawer + a children slot
/app/kpi/(kpi)/[...slug]/layout.tsx -> maybe not necessary unless there is layout that should reset on nav
/app/kpi/(kpi)/[...slug]/page.tsx

Would be very curious to understand if neither of these work for you

[ElectricCodeGuy]

I have this exact issue. But my Drawer list that i placed in the layout takes an ID as a param to render different list depending on what the user have chosen. So i gave up and moved the drawer back into the dynamic page again. The only solution is to restructure my database, but that will take way to long ^^
Would be nice if there was some sort of work around so the scroll position could persist and also take a param from the dynamic route

[gnoff]

@jkuhs Thanks for writing this proposal up. Our current thinking about why the behavior today makes sense is that usually when a dynamic segment changes the page represents a new object. Not keying the page can lead to security issues for instance retaining state about one account while updating the view to show a different account.

The use case of locale is somewhat unusual in that it isn't describing a new object, it is just contextualizing the page with some information that alters it's appearance generally. However the more common cases are that the dynamic segment means the identify of the page is distinct. If there are shared components in a page or layout that feel like they are not part of that identity then they can probably be lifted into a layout higher up that is not a child of the dynamic segment. Route groups can be useful if you want to lift up the layout but not share it with every other route at that higher level.

This is in contrast to search params which are not part of the key but are also not available in layouts, only pages. These can be updated without resetting the state of the tree but it also means that the searchParams can only be used "higher up" in client components where updates to the location can be reacted to without refetching the route.

So if we start from the perspective that components not directly tied to the identity represented by the dynamic segment should not remount when that dynamic value changes the first step is to see if that Component can be modeled as being part of a shared layout above the dynamic segment itself. I'd be curious to know if you have cases where this is not functional or practical?

this doesn't address the internationalization case though and it's something we have discussed internally and don't yet have a plan to communicate how we will address it but understand that we recognize it is not the same category of dynamic param use as most things are and it may require a more deliberate solution rather than making users choose to change dynamic param handling via config for instance. Your proposal opened with this example but it lays out a number of examples later that have more to do with blog posts. Was your motivation to open this primarily originating from the locale use case or did you encounter this in other contexts?

[sophiebits]

OK I thought about it a little more and I think I remember the constraints I ran into now.

The breadcrumbs that show the page's path are conceptually part of the layout, but they are determined by which path is being rendered within the layout. I wanted to map the path to a JS object describing the breadcrumbs, then pass that to the layout component so that it can render them.

In Next, I understand layout segments aren't refetched when navigating inside one so there is no way to pass props "up" the route hierarchy. So my temptation was to render the Layout component manually for each page, which does allow me to pass the breadcrumbs down. However then I run into the scroll state issue.

I understand that I can refactor the code to use parallel routes to render the breadcrumbs but then I can't pass the breadcrumb data as an arbitrary JS object; instead I need to refactor to render them as a standalone component, which is dissatisfying.

For example, what if each page had a "theme color" calculated by the page component that I wanted to pass to the layout component? I don't see a way to do this without imperative code controlling client-side state management (which doesn't work for SSR, etc).

[gnoff]

@jkuhs I appreciate your perspective but I will say that Next.js as a project does care about having secure defaults where possible and we don't consider this arbitrary. That doesn't mean we don't want to be useful in the cases that are currently bumping into this roadblock and we will consider what we can do here.

Do you mind expanding on this a bit more?

where we needed to dynamically render CMS pages through a single catch-all route
What's going on here that makes the use of a shared layout insufficient? Does the layout need parts of the page path when rendering? Is it a concern over fetching the CMS data from more than one place?

It seems a lot of the source of frustration with this issue relates to catchall cases both here in this disucssion and some of the linked issues provided by others.

Thanks for sharing your thoughts here. We appreciate it

[gnoff]

@sophiebits Ah that theme color example is a great one to really refine the limitation of hoisting shared UI into layouts thanks for bringing that up. It's kind of analogous to metadata where it is part of the page but to make it not block on layout rendering we expose this sort of parallel API to define it and for resuability reasons we make it embeddable in layouts. But semantically the metadata defined in a layout isn't owned by the layout it's still always owned by the page itself.

Theme color in this case is a property of the page but we have no way to compute that and make it available higher up (nor would we since we don't refetch shared layouts) and so you are now looking for a way to have layout-like experiences as part of a page. I'm not sure I have a good name to attach to this concept yet but it does feel distinctly different from a layout as I've come to think of it with respect to App Router.

I think you're right that there are a number of ways to do the breadcrumb thing that could work with a lifted layout (parallel routes is one but also just using a client component is another since it can read params) but it seems that the fact that it forces a particular code/data structure is frustrating. Probably because it feels like what you are trying to do is so simple so the forced complexity is resented (I'm editorializing here but trying to capture the essence of why this dynamic may lead to frustration)

[LavaToaster]

👋 Hey @gnoff

My intuition was that dynamic segments were a way of calling out to the next router to say hey you just need to worry about calling the main component, I don't need you worry about handling routing inside here, I'll deal with it. I don't know if I'm missing it, but the docs don't seem to be upfront about this behaviour.

That way when I implement a multi-step wizard form, I have full control over the routing and the urls shown in the navigation menu. This is because I want to support the end user being able to use back/forwards when navigating these form.

[mgreenw]

Really appreciate this conversation. Looking into this from a pages router -> app router migration perspective and this is really helpful / needed.

To add one more piece of (hopefully needed) complexity onto this "theme color" example: the properties of the page that need to be passed to the layout may themselves require data fetching. For example, the theme color may come from a CMS and may not be available during the first render of the page (if it suspends or shows a loading spinner). However, once the data becomes available, we want to make it available to the layout without fully re-rendering the layout and losing state.

I don't think it needs to be possible for this value to change dynamically once it has been fetched, but some sort of loading state is common for this page-specific data that needs to be available in layouts.

(Streaming) SSR is also potentially a tricky case for this "passing data up" case -- how would this handle the suspensions?

[cbratschi]

I provided my feedback in another discussion about the same issue:

#49749 (comment)

There are no workarounds for the use cases I listed. @jkuhs summarizes the issue and its side effects very well. Had exactly these issues.

[danvoyce]

This RFC is almost 10 months old and clearly has some strong arguments. Can we conclude this before it goes stale? What's the Next.js teams plans?

[sgoodrow-zymergen]

We should do this.

[SanderCokart]

I think for now at least I have basically come to terms with that the bridge between server and client has been built but it's missing some pillars and has collapsed in certain places.

I think for now I will have to accept that regarding the locale it is going to behave much like any other server rendered app except you do not see a hard reload.

I think the best ux experience would probably be to put the local switch in a place where a page does not contain stateful content.

This will at least prevent confusion or jarring layout shifts.

If somehow someday we can bridge this gap I will finally be a proud app router user.

But now I need to think about the existing apps in our company which are going to need i18n support.

Of course keeping in mind that for the average user the accept language headers are more than sufficient to detect the correct language and preventing the user from actually clicking on the locale switch button.

I don't know how often locals switch buttons are used and your websites or web apps I've never kept track of this metric.

[alexandernanberg]

I running into this issue as well.

I have a /posts/new route, when the user starts entering values I want to create the post and redirect to /posts/:id. Using router.redirect or redirect in the action will clear all local state, since the key changes. You can almost work around this with window.history.replaceState, but calling any server action after will cause the redirect and state reset regardless

[ElectricCodeGuy]

Overcoming Next.js' Search Params Limitation in Layouts
I read this article here witch seems like a possible solution. Creating a parallel route for the component that need to retain the state. I have not implemented it yet but for my case, retaining the position of the scroll position when navigating between catch all dynamic route seems like a good solution