Add dynamic routing

[amaury1093]

Feature request

Is your feature request related to a problem? Please describe.

As per #929, I am currently using next-routes for dynamic routing. However, it requires to have a custom server script, and creates some overhaul.

Describe the solution you'd like

Have it integrated in next.js by default, so that I don't need a custom server.js script, and I can run my dev server with next command.

Describe alternatives you've considered

Using a custom server.js, as described here: https://github.com/fridays/next-routes#on-the-server

Additional context

n/a

[NicoSartor]

We really need a better way of implementing routes with parameters.

I've been having a lot of issues setting up custom routes too. I tried next-routes, and nextjs-dynamic-routes, they all break something different in my rendering or in my routing.

[quantizor]

@timneutkens if you're thinking about routing changes at all for v7, I'd like to throw my library into the ring: https://buttermilk.js.org/

The support for parameterized routes is pretty strong and it's much much lighter than react-router, etc.

[mike-engel]

I personally like the idea behind this issue (and nuxt): #4890. That proposal would get me everything I need since, at least for my cases, it's never complicated parameterized routing.

[isBatak]

@timneutkens you showed some parameterized routing experiment in React Meetup presentation https://youtu.be/N-5TNKJ7eB0?t=4062, could we get some info about when it's going to land into next.js?
Also, will it be possible to render a child page inside a parent page, like it's done in _app with Component prop?

[timneutkens]

I can't share a timeline at this point.

will it be possible to render a child page inside a parent page, like it's done in _app with Component prop?

Most likely not, as this goes against having every page be a code-splitting entrypoint. _app.js wasn't introduced for layout capabilities but as a way to easier integrate apollo, redux etc.

[pragmaticivan]

I've been working with this approach in another library and has been very successful in production.

#5558

[seanconnollydev]

Not sure how far along you are at this point @timneutkens but I took a bit of time to see what it would take to implement a configuration based solution (similar to next-routes):

// next.config.js
module.exports = {
  routes: [
    {
      page: '/user',
      path: '/user/:id'
    }
  ]
}

https://github.com/goldenshun/next.js/blob/static-routes/examples/dynamic-routes/next.config.js

In that fork I was able to get dynamic routes working on the server with the above configuration. I have not yet implemented proper client routing, asset fetching, etc. I know you proposed something different in your React Meetup presentation but a file called [id].js reads awkwardly and moving files around if routes or parameters change is a bit heavy for my taste. I do like the idea of localizing route definitions to the page files themselves (as proposed in this Issue: #5558), but that has some downsides, which I commented on there. Exposing a fairly simple config in next.config.js seems like the best solution for parameterized routes I've seen thus fur.

[timneutkens]

We don't want a next-routes approach, it goes directly against the design constraint we had from the start: not shipping a definition of every possible route in the application. It simply doesn't scale, what if you have 100, 200 routes, or in case of zeit.co, 360 routes currently (and growing). Routes are unaware of each-other until they're loaded into the application by clicking <Link> / Router.push/replace.

[seanconnollydev]

It is possible to break route config files into chunks so they can be lazy loaded. For example:

/pages
  _routes.js
  index.js
  about.js
  /user
    _routes.js
    index.js
    profile.js

The root _routes.js file can be loaded by default.
When a <Link> or Router.push/replace encounters /user/*, it can fetch /user/_routes.js.

[martpie]

After a fairly large project with Next.js, I can share here that routing was the biggest pain point, I can explain things a bit more here.

it goes directly against the design constraint we had from the start: not shipping a definition of every possible route in the application

We had the content of the website on a headless CMS, and were using the pages like a templates directory. The problem came when some of the content had a link to another content item.

For example:

some content, blablabla <a href="cms:itemId:1q2w3e4r5t"> tada

To do the server-side rendering for this, we needed to transform these <a> to a <Link> with the correct query attributes, the correct asPath etc, and this is why we needed to know all the routes at runtime (we generate them when starting the app).

So the solution we used for is a big anti-pattern with Next.js default routing, but we did not find any better solution.

Today I am still trying to figure out a better way to do what we did. The problem was not really with the dynamic routing, but more with the <Link> component that is really hard to use for clean URLs with Route masking and dynamic content.

[Tarpsvo]

Hey @martpie

I'm currently struggling with the same issue. I wrote my own express server to handle all the routes and match the path names against the pages (templates) from the headless CMS. But I ran into a wall - Next.JS needs a map of all the paths during start-up (exportPathMap in next.config.js) to properly handle client-side routing. Without it, every time a user clicks a link, the whole page is reloaded to navigate them to the route. Did you somehow manage to work around this? Did you write your own Link component or Router?

If you could give me a little hint, I'd be really grateful. Thanks!

[martpie]

@Tarpsvo Sure, what we do is we build a map with all the pages of the application when starting the app and their parameters (asPath, query, and so on). It works for us because we know the content cannot be changed in production without redeploying (and then restarting the Next.js instances).

Then we have a React provider with this map in _app.

And we have a PageLink component that look like that (it's TypeScript), we give it an ID, and it returns a next/link with the correct props by consuming our provider to get the correct data from the generated map.

This PageLink component is used when generating the markup from the markdown stored in our headless CMS.

import * as React from 'react';
import Link from 'next/link';
import PagesLinks from '../../contexts/PagesLinks';

type Props = {
  pageId: string;
  prefetch?: boolean;
};

const PageLink: React.SFC<Props> = (props) => {
  const { pageId, children, prefetch } = props;

  return (
    <PagesLinks.Consumer>
      {pagesLinks => {
      // pageLinks is our hashmap
        const pageLinkData = pagesLinks[pageId]; // Careful, may be undefined

        return (
          <Link
            href={pageLinkData.href}
            as={pageLinkData.as}
            prefetch={prefetch}
            passHref
          >
            {children}
          </Link>
        );
      }
    }
    </PagesLinks.Consumer>
  );
};

export default PageLink;

This is a really painful solution, but the best we found so far.

[Tarpsvo]

@martpie That makes a lot of sense and I was leaning towards a similar solution. Thanks a lot! My other "solution" would've been to just use Razzle instead of Next.JS.

[idevelop]

Hey @timneutkens I just started learning next.js, loving it so far!

We don't want a next-routes approach, it goes directly against the design constraint we had from the start: not shipping a definition of every possible route in the application.

Correct me if I'm misunderstanding this, but isn't it suggested to do just that in the server side clean URLs tutorial, where the routes are all manually mapped in server.js? Isn't that effectively the same thing as next-routes, just more verbose?

I think that having support for rewriting as a built-in feature on the client-side (through <Link as="..." />) and requiring the dev to go lower level and create their own express server for server-side makes this feel like an incomplete aspect of the framework. I guess it depends on how strongly you believe that consistent URL rewriting across server-client should be a core part of the framework. Splitting the responsibility this way also creates the risk of inconsistent route definitions between server.js and whatever clean URLs are used in <Link />.

I was curious to learn from you but couldn't find the repo for the main zeit.co website: how do you guys define all the dynamic routes server side, for example for zeit.co/dashboard/project/:id?

I think this suggestion, with each route exported by the page, might be worth looking into: #5558

[revskill10]

I developed a PoC library to address the issue here . The beauty of my approach is that it works universally, not just for NextJS , but for all SSR framework out there.
Routing should be done universally, not frameworkful.

[ghost]

just started using next, dynamic routes need to be easier, better documented and standardized. If you google "nextjs dynamic routes" the first result is the useless "step by step" tutorial. I need docs and hello world examples. Imagine if you had to go through 5min of step by step and give out your email every time you need to google some api.

[thesunny]

@timsuchanek

What do you think about routes that instead of defining each path individually (i.e. a big file to ship that you want to avoid) instead configures route patterns to match to the file system?

Some benefits:

• The current lookup behavior is just a single route pattern. You don't need a breakout behavior for custom routes. The current behavior is the default route.
• Don't need to ship a large configuration of routes. Instead ship a few lookup patterns.
• A single route pattern can match against hundreds or thousands of files. Allows next.js to lazy load pages without needing a large config.

Here's an example of what Next.js current route might look like:

// next.config.js
module.exports = {
  routePatterns: [
    /**
     * Matching an unlimited number of paths deep in the file system and
     * similar to the way next.js handles by default
     *
     * localhost/hello
     *
     * file: '/hello.js'
     * params: []
     */
    {
      urlPath: "/$$", // $$ signifies matching more than one level deep in file system
      filePath: "/", // signifies that it's the root of /pages
    },
  ],
}

Here's some routes to match something like github.com:

// next.config.js
module.exports = {
  routePatterns: [
    /**
     * Matching a github user/team profile. Exactly one segment.
     *
     * github.com/zeit
     *
     * file: '/profile.js'
     * params: ['zeit']
     */
    {
      urlPath: "/*",
      filePath: "/profile",
    },
    /**
     * Matching a github project. Exactly two segments.
     *
     * github.com/zeit/next.js
     *
     * file: '/project.js'
     * params: ['zeit', 'next.js']
     */
    {
      urlPath: "/*/*",
      filePath: "/project",
    },
  ],
}

Then let's say we wanted to add some admin features and utils which can map hundreds/thousands of files in a small config:

// next.config.js
module.exports = {
  routePatterns: [
    /**
     * Matching a (non-existent) set of hundreds of github admin tools.
     * Any number of segments but matching files one level deep with the
     * remaining segments passed in as params.
     *
     * github.com/admin/make-toast/instant
     *
     * file: '/admin-tools/make-toast.js'
     * params: ['instant']
     *
     * or
     *
     * github.com/admin/make-toast/instant/dark
     *
     * file: '/admin-tools/make-toast.js'
     * params: ['instant', 'dark]
     */
    {
      url: "/admin/$/**", // two stars maps any number of params
      path: "/admin-tools",
    },
    /**
     * Matching an unlimited number of paths deep in the file system and
     * similar to the way next.js handles by default
     *
     * github.com/utils/make/me/a/sandwich
     *
     * file: '/utils/make/me/a/sandwich.js'
     * params: []
     */
    {
      url: "/utils/$$", // two stars maps any number of file paths deep
      path: "/utils",
    },
    /**
     * Matching a github user/team profile. Exactly one segment.
     *
     * github.com/zeit
     *
     * file: '/profile.js'
     * params: ['zeit']
     */
    {
      urlPath: "/*",
      filePath: "/profile",
    },
    /**
     * Matching a github project. Exactly two segments.
     *
     * github.com/zeit/next.js
     *
     * file: '/project.js'
     * params: ['zeit', 'next.js']
     */
    {
      urlPath: "/*/*",
      filePath: "/project",
    },
  ],
}

[thesunny]

In case it wasn't clear:

• $: matches one segment and adds it to the filePath to find the correct file to load
• $$: matches one or more segments and adds it to the filePath to find the correct file to load.
• *: matches one segments and adds it to the params
• **: matches zero or more segments and adds it to the params

One trade off is that we don't have named params only an array of params. It would be difficult to get named params without shipping a whole bunch of routes. I don't think it's a bad trade-off.

[miketdonahue]

@timneutkens to the point of @idevelop, how is ZEIT accomplishing dynamic universal routing for the main zeit.co website with 360+ routes?

[martpie]

@meebix

• https://nextjs.org/learn/basics/clean-urls-with-route-masking
• also, a lot of pages have hardcoded content: there is no /blog/:slug, there is an individual page for all posts.

In a real world example, your would have a dynamic routes, a CMS, and user-generated content with links to other pages of your website, which is really hard to do with route masking.

In the case of Zeit, they unfortunately don't face the same problem than us, so this may explain why it is not as important for them as it is for us in their backlog. (Typically, I don't care about AMP, but I care a lot about dynamic routing, and especially a simplified <Link> component)

Edit: I'll add one thing, I think the main confusion comes from the meaning of "pages": are they actual pages, or are they routes?

[timneutkens]

In the case of Zeit, they unfortunately don't face the same problem than us, so this may explain why it is not as important for them as it is for us in their backlog. (Typically, I don't care about AMP, but I care a lot about dynamic routing, and especially a simplified component)

• We have dynamic routing in zeit.co, probably more complex dynamic routing than the average app as we have catchalls.
• We are working on dynamic routing right now. Contrary to what you may think we've been thinking about this issue for a really long time and it's much more nuanced than "just implement it"
• Note that we only get one shot at introducing APIs, after that we have to support it till the end of time.

In the case of Zeit, they unfortunately don't face the same problem than us

I talk to large and small users of Next.js every single day. Even in weekends. I'm fully aware of all the feedback there is around every single issue that we have, and we're continuously solving these more complex issues.

My DMs are open on both spectrum and twitter:

• https://twitter.com/timneutkens
• https://spectrum.chat/users/timneutkens

[martpie]

@timneutkens Thanks for the clarification. My point was really not to bash Next.js (it's definitely the best React framework out there), but to explain that the problems you were initially solving at Zeit may be different than the ones other people from the industry face today (typically, CMS content, we had this discussion on Twitter already), of course, it changes with time. But I'm happy to see I'm wrong :)

I talk to large and small users of Next.js every single day. Even in weekends.

I think no one can contest that :)

Excited to see how you'll manage to solve this complex problem.

[typeofweb]

@timneutkens wow, great to hear that! I appreciate all the hard work you people are doing. Next.js is powering my latest app and I couldn't possibly imagine a better SSR framework.
Do you have a draft spec of the routing you're working on? Or maybe some proof of concept or ideas?

[typeofweb]

Found it, just published: #7297

[timneutkens]

@mmiszy #7297 is not related to dynamic routing. That's the next proposal I'm writing.

[martpie]

Would advanced routing patterns/regex like { page: 'post/list', pattern: '/posts/directory/:letter([a-z])?' } be supported by the potential new $ notation @timneutkens? Or should we validate the query in getInitialProps and res.send a 404 if needed?

[timneutkens]

Yeah initially just validating in getInitialProps, we might extend the syntax later on, however initially I'd like to keep it simple.

[timneutkens]

Closing this because of #7607

[balazsorban44]

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.