Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃挕 RFC: SSR #80

Closed
FredKSchott opened this issue Apr 12, 2021 · 28 comments
Closed

馃挕 RFC: SSR #80

FredKSchott opened this issue Apr 12, 2021 · 28 comments

Comments

@FredKSchott
Copy link
Member

FredKSchott commented Apr 12, 2021

Our current intention is to launch Astro as a static site builder. That means all pages are built to static HTML, and no support for dynamic server-side routes.

But, there seems to be a lot of community interest in supporting dynamic routes & pages. If there's enough interest and anyone willing to help us with an implementation, then I'd love to start putting together an RFC for experimental support. If we put this behind an experimental flag to start, then I don't think it interferes with our "static site builder" launch story.

Anyone interested in helping spec out an RFC?

Some pieces that need discussion / fleshing out:

  • Dynamic pages: can we borrow from Next.js's file based routing? /pages/[id].js, /pages/[...id].ks, etc.
  • Dynamic props: how does a page register as dynamic? Something like getServerSideProps?
  • Deploy targets: In addition to a Node.js server deployment, I really want us to have a good Cloudflare Workers story. SvelteKit too the approach of having different "adapters" for Node, Deno, etc. Could we do the same?
@matthewp
Copy link
Contributor

matthewp commented Apr 12, 2021

Can you explain the second bullet point a little more? I don't follow what you mean by "register as dynamic". Thanks!

Looked into getServerSideProps and I think that's a way to load asynchronous data before a page component is run. We have this in the form of "top level await" already. So yeah, I think I'm misunderstanding that bullet point. Would love some clarity.

@matthewp
Copy link
Contributor

To help standardize across hosts I think we should make the following changes:

  • The current request.* properties related to the URL should because request.url and be a URL object.
  • Add request.headers which is a headers object (Deno supports this as a global, environments that don't there's a impl in the fetch library).

@jonathantneal
Copy link
Contributor

Please forgive me for not knowing where y鈥檃ll might be collaborating on an RFC. In the meantime, I鈥檒l share an agnostic take on the patterns I鈥檝e observed from Next and similar tooling. I鈥檒l limit this take to html files, but I expect you can lift this to astro files if it is at-all useful.


File-System Routing

When using file-system routing, each file in a specified pages directory represents a route.

Static routes are represented by file paths relative to the pages directory.

Additionally, static directory routes (routes that might end with /) are represented by index files.

File Route
./pages/index.html /
./pages/about.html /about
./pages/blog/index.html /blog/
./pages/blog/first-post.html /blog/first-post

Dynamic routes

Dynamic routes are represented by file paths that include square brackets.

Within a segment of a file path, a term between square brackets ([param]) captures shallow routes. Adding three dots before a term ([...params]) captures deep routes (which may include shallow routes).

File Pattern Example Request
./pages/blog/[slug].html /blog/* /blog/doing-the-web
./pages/post/[...all].html /post/** /post/1984/mac/i386
./pages/[username]/settings.html /*/settings /joebloggs/settings

The term between the brackets is used to identify any dynamically matched segment or segments.

Filename Example Request Example Identifiers (JSON)
./pages/post/[...all].html /post/1984/mac/i386 { "all": [ "1984", "mac", "i386" ] }
./pages/[username]/settings.html /joebloggs/settings { "username": "joebloggs" }

Static routes take precedence over dynamic routes. For dynamic routes, shallow routes take precedence over deep routes.

Filename Pattern Example Request
./pages/blog/[slug].html /blog/* /blog/doing-the-web
./pages/post/[...all].html /post/** /post/1984/mac/i386
./pages/[username]/settings.html /*/settings /joebloggs/settings

Single brackets ([param]) do not capture empty segments (like their own directory route). Double brackets ([[param]], [[...all]]) also capture empty segments. Single brackets take precedence over double brackets.

Consider the following file tree:

./pages/
鈹溾攢鈹 post/
鈹   鈹溾攢鈹 create.html
鈹   鈹溾攢鈹 [pid].html
鈹   鈹斺攢鈹 [...slug].html
  • ./pages/post/create.html matches /post/create.
  • ./pages/post/[pid].html matches /post/1, /post/abc, etc, but not /post/create, /post/1/2, etc.
  • ./pages/post/[...slug].html matches /post/1/2, /post/a/b/c, etc, but not /post/create, /post/1, etc.
  • Optional, "catch all routes" would be defined using double brackets and dots.

Note: When file-system routing is paired with static assets (in separate pages and public directories), static assets take precedence over file-system routing.

@duncanhealy
Copy link
Contributor

duncanhealy commented Apr 12, 2021

Cloudflare links

@matthewp
Copy link
Contributor

matthewp commented Apr 12, 2021

@jonathantneal Thanks so much for writing that up. I think that's almost exactly what we want (imo)! One additional type of route that we're currently working on is Collections routes. It's described in this PR: #68

It allows for paginated behavior without needing full dynamic support (paginated routes work in static rendered builds whereas dynamic likely won't).

@FredKSchott FredKSchott changed the title [RFC] Dynamic Pages [RFC] Dynamic Routes Apr 12, 2021
@FredKSchott
Copy link
Member Author

Can you explain the second bullet point a little more? I don't follow what you mean by "register as dynamic". Thanks!
Looked into getServerSideProps and I think that's a way to load asynchronous data before a page component is run. We have this in the form of "top level await" already. So yeah, I think I'm misunderstanding that bullet point. Would love some clarity.

Astro needs to know at build time: is this a buildable static page, or a dynamic runtime-only page? Static pages would be built to HTML, and dynamic pages would be built to JS to run in a server context.

@matthewp
Copy link
Contributor

matthewp commented Apr 12, 2021

What's the reason for compiling a static page to HTML? Perf? As we've discussed previously, if we change the compilation to strings we're talking about a "dynamic" page module for a static page as being something like this:

function __render() {
  return "<html><head><title>My app</title></head><body><h1>My app</h1></body><html>"
}

You can't get much better than that in terms of perf. So if that's the reason I don't think it's worth the added complexity of having 2 different modes and forcing the user to decorate their pages (which they might do incorrectly).

@jonathantneal
Copy link
Contributor

Inspired further by your existing collections work and these comments, I am wondering, could dynamic routes be expressed more simplistically in the file name, and then more expressively in their front matter?


Commence Bikeshedding

File paths with dollar-prefixed segments define static routes.

File paths with double-dollar-prefixed ($$) segments define server routes.

Would the $$ qualification immediately limit the number of directories and/or files prepared to "static all the things"?

File Request Params
./pages/photo/$id.astro id (static)
./pages/photo/$id/edit.astro id (static)
./pages/photo/$$query.astro query (dynamic)
./pages/$$user/dashboard.astro user (dynamic)
./pages/post/$pid/$comment.astro pid (static), comment (static)

Additional routing restrictions are optionally exported from front matter as a method. The method receives a request object and returns a truthy value if the request should be further handled by the file.

// ./pages/photo/$$query.astro
---
export const onRequestFilter = (req) => (
  req.method === 'POST'
  && req.params?.query.length > 1 // note: req.params !== req.url.params
)
---

Conclude Bikeshedding


If acceptable, the server would have two-step filtration; the first being the more simplistic, file-system-generated filter; and the second being the more expressive onRequestFilter method exported from front matter.

@eyelidlessness
Copy link
Contributor

This is probably another case that would benefit from copying prior art. Many similar products have had success just adopting Next/Gatsby APIs wholesale. This is not only good from the perspective of benefitting from those teams鈥 R&D, it also enables people like me to feel more comfortable pitching less established frameworks to clients who might be hesitant.

In that spirit鈥攁nd I know this is going beyond the scope of this particular issue, but I think it鈥檚 worth discussing holistically鈥擨 would like to propose:

  1. Revise .astro, the template format, to just be JSX/TS with a different file extension, or to be a configurable static-only format deferring to a user鈥檚 renderer of choice. (This could also be handled by the convention discussed in RFC: renderer plugins聽#74, as eg .astro.jsx etc).

  2. Adopt Next鈥檚 APIs and filesystem patterns for pages, or the most common APIs in the user鈥檚 preferred ecosystem.

This would be a huge win for folks who may want a near drop in replacement for their not-as-static-as-advertised SSG/SSR framework. Imagine being able to say 鈥渟witch from Next to Astro and cut your page weight 75%. It only takes a day.鈥

@matthewp
Copy link
Contributor

@eyelidlessness We need the .astro file format in order to statically analyze a template, to know what parts are always static and which are dynamic (just expressions and components). Adopting JSX means everything is dynamic and we lose a big part of what makes Astro work.

@eyelidlessness
Copy link
Contributor

I'm not sure I understand. JSX is also statically analyzable. The mechanism used in .astro to determine whether a given component should be dynamic (currently :load etc suffixes, possibly as props pending #73) could be used in JSX as well, using estree or your AST tooling of choice. This would reduce some burden on Astro (no need to maintain yet another template syntax), as well as cognitive load on users (no need to know yet another template syntax or use two in the same project).

@jonathantneal
Copy link
Contributor

鈥渟witch from Next to Astro and cut your page weight 75%. It only takes a day.鈥

In am in no way attempting to judge your React or NextJS experience, but both of these projects have continued to evolve in significant ways, multiple times; including what many consider "paradigm shifts" in how one鈥檚 code base for a site are authored; and in particular with NextJS how significant aspects of file system routing are authored.

NextJS is fantastic! Even then, one could spend more than a day migrating a reasonable site from some older version of NextJS to some newer version. "I was there, Gandalf."

But their evolution is a feature; not a bug. This is to say 鈥 if Astro finds a better way to do something, and they ship it, and it gains reasonable traction, then their better way will be become the defacto way others want to follow, until someone finds an even better way again.

@matthewp
Copy link
Contributor

Maybe we're talking about different things, but what i mean is that in a JSX file you can do:

export let render;
if(Math.random() < 0.5) {
  render = () => <div>Like this</div>
} else {
  render = () => <span>Totally different</span>
}

JavaScript is just completely dynamic, we chose a static template format for the same reasons as Svelte.

@eyelidlessness
Copy link
Contributor

@jonathantneal

In am in no way attempting to judge your React or NextJS experience, but both of these projects have continued to evolve in significant ways, multiple times; including what many consider "paradigm shifts" in how one鈥檚 code base for a site are authored; and in particular with NextJS how significant aspects of file system routing are authored.

This is fair, and surely could be a burden to maintain. Although at least in the last few versions of Next it's seemed to have mostly additive changes to an otherwise relatively stable API. So much so that it's quite common to find getStaticProps, getStaticPaths, getServerSideProps etc in comparable tools.

But their evolution is a feature; not a bug. This is to say 鈥 if Astro finds a better way to do something, and they ship it, and it gains reasonable traction, then their better way will be become the defacto way others want to follow, until someone finds an even better way again.

This is possible, for sure. And I wouldn't suggest a better way might not be found, here or anywhere. I'm just saying it's been beneficial to other projects to adopt some or all of Next's APIs.

@matthewp

Maybe we're talking about different things, but what i mean is that in a JSX file you can do:

export let render;
if(Math.random() < 0.5) {
  render = () => <div>Like this</div>
} else {
  render = () => <span>Totally different</span>
}

JavaScript is just completely dynamic, we chose a static template format for the same reasons as Svelte.

Sorry for asking instead of trying it out first, I'm not somewhere I'd be able to try it out at the moment: isn't that possible in .astro as well? It certainly seems it could be. The frontmatter-style heading appears to accept arbitrary JS/TS, as well as arbitrary imports from JS/TS. And it appears to accept JSX-style conditionals in template expressions. I'm still unclear on how static analysis of .astro avoids this problem, but open to the possibility I'm missing something.

@matthewp
Copy link
Contributor

matthewp commented Apr 13, 2021

@eyelidlessness Let's take a step back for a moment as I think discussing technical details misses a larger reason for why .astro files look the way they do. When we started working on Astro our goal was to provide a way to write mostly HTML and sprinkle in components where you need them. Initially we weren't going to have Astro components at all. We didn't have expressions at all. The prototype for Astro used a regular HTML parser.

The .astro file format was never the goal, not the thing we wanted to try and sell. It evolved where it made sense to. I'm still hesitant about having complex expressions inside of .astro files, personally. To me that's what components provide. I say this all to say, we didn't set out focused on making a competitive component format. We set out to let you write HTML and components in the framework of your choice.

Switching to JSX I think would harm that goal. You'd be dropped into a dynamic programming language where you have to learn a bunch of special rules. Those rules are familiar to people coming from Next.js, but they're not if you're coming from anywhere else. It would also, I think, be a turnoff for Vue or Svelte or other framework users not already bought into JSX. We don't want to be a clone of Next.js with a few differences.

@duncanhealy
Copy link
Contributor

prior art also includes vitedge c.f. https://github.com/frandiox/vitessedge-template
for cloudflare it is using webpack with config imported from node_modules folder

@karolis-sh
Copy link

For bigger, more complex sites SSR capability is pretty much a must (hundreds of pages, cookie personalization, constant headless CMS content changes). Generate performant pages + cache them via CloudFlare = 鉂わ笍
Next.js' API seems pretty well thought out, though it's a SSR by default and static HTML export is a secondary flow.

@philippe-elsass-deltatre
Copy link

philippe-elsass-deltatre commented Jun 9, 2021

I would like to +1 here that, as with many SSR solutions, routing is again "static":

  • How do I make a multilingual website with translated routes?
  • can I bring my own routing logic? our projects use a dynamic routing mapping logic that we load server-side and client-side, where each route is mapped to a rendering template

@FredKSchott FredKSchott added this to Feature Request in 馃悰 Bug Tracker Jun 13, 2021
@FredKSchott FredKSchott changed the title [RFC] Dynamic Routes 馃挕 RFC: Dynamic Routes Jun 30, 2021
@rebelchris
Copy link
Contributor

+1 from my side, I realised this actually works on dev build.
So I would indeed think it could work something like Next.js JIT rendering?

I think all we would need is the params variable not?

file: $tweets.astro
builds: tweets/:tweet (singular)?
on page: $tweet is the ID?

I would hope there we can use this dynamic ID as a JIT rendering to retrieve some API details.

@askeyt
Copy link

askeyt commented Aug 19, 2021

Hi all,

Very new to Astro and apologies if this is not the right place for this. But bear with me and please consider the following:

---
// $pages.astro
export async function createCollection() {

  // Flattened page list
  const items = [
    {
      name: 'My Title',
      slug: 'my-title',
      fullSlug: 'my-title',
    },
    {
      name: 'My Child',
      slug: 'my-child',
      fullSlug: 'my-title/my-child',
    },
    {
      name: 'My Child 2',
      slug: 'my-child-2',
      fullSlug: 'my-title/my-child-2',
    },
  ];

  return {
    route: `/pages/:fullSlug`,
    paths() {
      return items.map((page, i) => ({params: {fullSlug: page.fullSlug}}));
    },
    async props({ params }) {
      return {item: items.find((page) => page.fullSlug === params.fullSlug)};
    },
  };
}
const {item} = Astro.props;
---
  1. http://localhost:3000/pages/my-title works :)
  2. http://localhost:3000/pages/my-title/my-child doesn't work :(

The error I get for 2 is:

Error: [createCollection] route pattern does not match request: "/pages/:fullSlug". (/pages/my-title/my-child)

Now, coming from a Vue background which uses path-to-regexp (as do you?).. I tried changing the returned route property in createCollection to route: /pages/:fullSlug+ (see here: https://github.com/pillarjs/path-to-regexp/tree/v1.7.0#one-or-more)

Now I get the following error:

TypeError: Expected "fullSlug" to match "[^\/#\?]+?", but got "my-title/my-child"

Now for the reason of my post

By extending the behaviour of the route param to follow all the rules of path-to-regexp, does this not solve the problem of "dynamic routing" ? Everything comes down to mapping the collections and the props with the route.

There would need to be rules around which takes precedence, static files vs regex patterns, and I'm sure I'm not taking into account a lot of things 鈥 but no one has mentioned this here yet and I thought it was worth doing so.

@matthewp
Copy link
Contributor

馃挕 RFC Tracker [No Longer Used] automation moved this from On Hold to Completed Aug 23, 2021
@FredKSchott FredKSchott changed the title 馃挕 RFC: Dynamic Routes 馃挕 RFC: SSR Aug 23, 2021
@FredKSchott
Copy link
Member Author

I got confused by this too! But I think this issue was originally designed to track what we now just call SSR. Reopening to keep gathering feedback

@FredKSchott FredKSchott reopened this Aug 23, 2021
@askeyt
Copy link

askeyt commented Aug 23, 2021

This is out! https://docs.astro.build/core-concepts/routing#dynamic-routes Closing

This sorts me out! Thanks

@CEbbinghaus
Copy link

CEbbinghaus commented Oct 1, 2021

This would be a wonderful feature. Ideally allowing for things like Catchall Routes that result in an Error404 page and allowing for content to update dynamically as it gets added.

What's the reason for compiling a static page to HTML? Perf? As we've discussed previously, if we change the compilation to strings we're talking about a "dynamic" page module for a static page as being something like this:

function __render() {
  return "<html><head><title>My app</title></head><body><h1>My app</h1></body><html>"
}

You can't get much better than that in terms of perf. So if that's the reason I don't think it's worth the added complexity of having 2 different modes and forcing the user to decorate their pages (which they might do incorrectly).

I feel like this or something akin to it would be a great compromise. Having half static and half dynamic pages would require whatever SSR code is running to do additional checks when having to retrieve static content.

I think Astro and in particular, .astro files lend themselves uniquely to SSR being logically split into the "build" and "render" steps. Allowing the "build" step to happen when the request fires with top-level await would make for some very elegant solutions to resolving path information and similar.

@tbeseda
Copy link
Contributor

tbeseda commented Oct 1, 2021

Wild idea: what if Astro wasn't a full server framework? Not all of one, anyway. Just1 make it mountable in existing frameworks.

If Astro's runtime could be mounted, it wouldn't need to worry about higher level routing, middleware (like auth), static assets (including .html pages generated by Astro at build time), sessions, etc. Astro's theoretical server runtime could accept the standard request/response args and return HTML

Astro would exist somewhere between a full web framework and a view engine. It could "route" the request in that Astro would pick which .astro file to load up and render. But it would do this after the framework in charge has already handled the rest.

Hypothetical: "螞stroLive"
I set up my main server.js with the framework du jour. The framework parses the request, auths the user, populates the session from the db, etc. Then it also mounts 螞stroLive: myApp.use('/app', Astro.live('./path-to/astro-views', options))2
Now I can write most of the fun parts of my app in Astro files! They have access to the original request object with things like session already populated by my framework. And the Astro files always return HTML.

I have no idea how this works for things like partial hydration. But Astro could be responsible for setting up internal paths for those resources at the root I set in the framework (/app in the example above).

A key advantage, aside from not having to build and maintain a full JS web framework: it would be great for serverless.

馃 Maybe this is a separate idea. or a convenient half way step to the ultimate goal.3

4

Footnotes

  1. I say "Just", but I realize it's still a huge effort.

  2. I made up 螞.live() -- but I bet there's a cool space-y name someone more clever than I can come up with.

  3. If this is not in line with the goals of Astro, I won't mind. just something I've been thinking about.

  4. GitHub Markdown footnotes are awesome, btw.

@jasikpark
Copy link
Contributor

Wild idea: what if Astro wasn't a full server framework? Not all of one, anyway. Just1 make it mountable in existing frameworks.

If Astro's runtime could be mounted, it wouldn't need to worry about higher level routing, middleware (like auth), static assets (including .html pages generated by Astro at build time), sessions, etc. Astro's theoretical server runtime could accept the standard request/response args and return HTML

Astro would exist somewhere between a full web framework and a view engine. It could "route" the request in that Astro would pick which .astro file to load up and render. But it would do this after the framework in charge has already handled the rest.

Hypothetical: "螞stroLive"

I set up my main server.js with the framework du jour. The framework parses the request, auths the user, populates the session from the db, etc. Then it also mounts 螞stroLive: myApp.use('/app', Astro.live('./path-to/astro-views', options))2

Now I can write most of the fun parts of my app in Astro files! They have access to the original request object with things like session already populated by my framework. And the Astro files always return HTML.

I have no idea how this works for things like partial hydration. But Astro could be responsible for setting up internal paths for those resources at the root I set in the framework (/app in the example above).

A key advantage, aside from not having to build and maintain a full JS web framework: it would be great for serverless.

馃 Maybe this is a separate idea. or a convenient half way step to the ultimate goal.3

4

So something with the similar energy of https://tokio.rs/blog/2021-07-announcing-axum where all the middleware is just express or ember or something?

Footnotes

  1. I say "Just", but I realize it's still a huge effort.

  2. I made up 螞.live() -- but I bet there's a cool space-y name someone more clever than I can come up with.

  3. If this is not in line with the goals of Astro, I won't mind. just something I've been thinking about.

  4. GitHub Markdown footnotes are awesome, btw.

@chriscalo
Copy link

chriscalo commented Mar 16, 2022

This issue was closed when dynamic routes were released, but this seems to be just a mechanism for generating multiple static .html files.

"adapters" for Node, Deno, etc.

What happened to the idea of a server-side runtime? The Astro programming model feels like a big improvement over what came before it, I just wish it wasn't a static-build-only tool.

Is the team still open to the idea of a server runtime? Is this tracked in another issue?

@may17
Copy link

may17 commented Jul 3, 2022

@chriscalo see here https://docs.astro.build/de/guides/server-side-rendering/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

No branches or pull requests