Parameterized routing with filesystem for #91

[nodegin]

Hello, here is a proof of concept pull request for issue #91 that achieves parameterized routing using only filesystem structure.

Filesystem

Code

import React from 'react'

export default class extends React.Component {
  static async getInitialProps ({ req, params }) {
    // params property must matches filename
    return { test: params.id }
  }

  render () {
    return (
      <b>{ this.props.test }</b>
    )
  }
}

Result

[CompuIves]

Interesting PR! I like the idea very much, although I do have a concern that we're now deviating a bit from the way how nested routing was done in Next (exclusively using folders). I personally still prefer the idea with files like _id.js (or in this case @id.js, which seems better because it's more explicit) as initially proposed in #91.

Still, I really like the fact that we now have a working version of parameters in a file system! 👍

[richardnias]

I agree with @CompuIves, something that uses nested directories feels better. I was working on something similar to this, which had files called :id.js, which would be placed in a users folder, rather than files like users_:id.js or users_@id.js. It was a really simple implementation (when resolving an URL to a filepath, as well as /my/path/index.js and /my/path.js, look for /my/:id.js). I haven't submitted a PR because it had shortcomings:

• the param had to be :id.js (easily solved to be sure)
• no support for routes like /users/:userId/posts/:postId

But both of these could be relatively easy to overcome - probably by taking a more advanced approach to changing the resolver than what I did.

[nodegin]

@nias-r I have tested with this PR which supports /users/:userId/posts/:postId, just name the filename like users_@uid_@action_@pid where params.action would be posts.

[timoxley]

I'd suggest keeping the mapping between files and routes as straightforward and non-magical as possible.

@nodegin is there a reason for using @ over :?

This means no users_@id.js -> users/:id magic, just keeping a strict mapping of routes to files/directories, via node/webpack's default module resolution algorithm.

i.e.
/users/:userId -> pages/users/:userId.js or pages/users/:userId/index.js

The second form with the pages/users/:userId directory + index.js would be useful in cases where you're nesting beyond a :param

> tree ./pages
./pages
├── index.js
└── users
    └── :userId
        ├── index.js
        └── posts
            └── :postId.js

[nodegin]

@timoxley someone point out that : might be not work properly on some systems.

[jharmn]

There is a nice implementation of directory-style routing, similar to @timoxley 's suggestion, which I've used for APIs in swaggerize-express (see Path Routing), originally written by @tlivings:

  |--users
  |    |--{id}.js
  |    |--{id}
  |        |--foo.js

Note the use of {id} for parameters, which is a much more common convention (and avoids the : issue).
In the original PR, the mapping of @ and _ to params and directories seems a little arbitrary and easy to confuse.

[giacomorebonato]

I see a problem in naming files like {id}.js when you start to have a lot of file with the same name in a project and it is not comfortable anymore to "jump" to the one you want.

[timoxley]

@giacomorebonato nobody stopping you adding context e.g. {userId}.js

[CompuIves]

I really like the idea of using {} instead of _. Much more explicit. Perfect scenario would be to do the initial proposal of #91 (like explained by @jharmn) but with {} instead of _.

Do you want to implement this @nodegin? I can do this as well if you're busy.

[mattapperson]

This looks great! But I agree with @CompuIves. I might suggest an alternate of ${} as this fits the js pattern of vars in strings as of es6

[tlivings]

@mattapperson the purpose of {someParam} is for consistency with swagger path definitions (not to mention hapi).

There's nothing wrong with ${} but why bother matching es6 template literals or having an additional character?

[mattapperson]

@tlivings I dont disagree with you at all, but this project hides hapi, and it's not swagger. It is a server side react server. By it's nature it is JS first. the most JS way to do this in my mind is to use Template literals style.
That said I have NOTHING against {someParam}, ${someParam} just feels a little more pure to me.

[tlivings]

No, I get it — I just referenced the other projects because that may also be what people are more familiar with.

[mattapperson]

@tlivings I feel ya! Another counter point to my argument though would also be variables in JSX are {param name} so....

[dstreet]

Does anyone know of an existing system using a filesystem api like this?

[timoxley]

@tlivings @mattapperson template literal syntax is a non-starter due to shell variable expansion.

e.g. try creating a file named ${id}.js or ${USER}.js in your terminal.

[mattapperson]

Hah good point @timoxley!

[timoxley]

@dstreet see above comment about swaggerize-express.

[dstreet]

@timoxley Thanks. Not sure how I missed that

[nodegin]

Hi guys, I have updated this PR to use the format as @jharmn suggested:

  |--users
  |    |--{id}.js
  |    |--{id}
  |        |--foo.js

But I found a bug that causing HMR function to be broken when the link is in /user/{id}/foo format, but not /user/{id}. Help needed for investigating..

[mattapperson]

@nodegin I suggest/request this gets merged and a new ticket get created for the known HMR bug. This is a major feature that allows next.js to be used in much larger apps. The HMR bug is annoying but not critical. This feature however is critical to many apps (like mine, selfishly)

[nodegin]

@mattapperson Only collaborators can merge this PR

[mikecardwell]

Using this branch, I created a "pages/index.js". Requests to "/" are routed to that page. I then created an additional "pages/{id}.js". Now requests to "/" are being routed to "pages/{id}.js" even though pages/index.js is a more specific match.

Also, requests for "/foo/bar" are also getting routed to "pages/{id}.js" - This is not how other frameworks handle this type of routing. I would expect "/foo" to get routed to "pages/{id}.js", but "/foo/bar" to 404.

[mikecardwell]

Also. I created "pages/foo/index.js", and now requests for "/foo/" get routed there, but "/foo" get routed to "pages/{id}.js".

[kawaz]

I think that it is not good to use parameter names for file names and paths. Because it causes you to do ls to find _xxx.js. For example, in a directory containing 100,000 files, performance will be problematic.

I think that it is better to statically determine the name of the file to search from the request URL. I have an idea.

  |--users
  |    |--_params.js ... `export default { path: "{id}", name: "id" }` 
  |    |--{id}.js ... this file name determ statically by definition of _params.js, don't need ls.
  |    |--{id}
  |        |--foo.js

Developers can freely determine parameter names and path names.

  |--users
  |    |--_params.js ... `export default { path: "_id", name: "userId" }` 
  |    |--_id.js ... url is /users/:userId
  |    |--_id
  |        |--foo.js ... url is /users/:userId/foo

And catch all parameter.

  |--users
  |    |--_params.js ... `export default [ { path: "_id", name: "userId", pattern: /^[0-9]+$/ }, { path: "_actions", name: "action" } ]` 
  |    |--me.js ... url is /users/me
  |    |--_actions.js ... url is /users/:action
  |    |--_id.js ... url is /users/:userId
  |    |--_id
  |        |--foo.js ... url is /users/:userId/foo
  |-- files/_params.js ... `export default { path: "download", name:"path", pattern: /.+/ }`
  |-- files/download.js ... url is /files/:path ( path matches greedy )

• /users/me can find statically.
• /users/123 can not find.
  • /users/_params.js is found, and match first pattern.
  • path is /users/_id, and /users/_id.js is found.
• /users/happy can not find statically.
  • /users/_params.js is found. and match second pattern.
  • path is /users/_actions, and /users/actions.js is found ( The default of pattern is /[^\/]+/ )
• /users/123/foo can not find.
  • /users/123/ can not find.
  • /users/_params.js is found. and match first pattern.
  • path is /users/_id, and /users/_id/foo.js is found.
• /files/foo/bar/baz.jpg can not find.
  • /files/foo/bar/ can not find.
  • /files/foo/ can not find.
  • /files/ can find.
  • /files/_params.js is found, and matched.
  • path is /files/download, /files/download.js is found. params.path is foo/bar/baz.jpg.

[kawaz]

My performance problem (many files in directory) is only in development. (with hot loading)
But if next build can make static routing rule, the problem can resolve?

[timoxley]

@nodegin bump, looks like there's some conflicts to fix

[nodegin]

@timoxley I believe this PR is out of date and Next.js probably would go for another better solution such as a routes.js. However if there's anyone wanna help to fix these conflicts are welcome to reach me by leaving the username for being collaborator in my forked repo.

[tlivings]

@nodegin can you provide details on the HMR bug?

[mikecardwell]

There are also the bugs that I mentioned. I don't think this is just a case of fix the conflicts and then release, IMHO the numerous bugs need fixing first.

[kachkaev]

The conversation was moved to #291 (if I'm not wrong - just a newbie here)