Reroute API

[lilnasy]

Body

• Exploration branch: lilnasy/proposal-reroute lilnasy/proposal-reroute-new-impl

Summary

Programmatic control over routing.

Example

Astro Page

---
return Astro.reroute("/404")
---

API Route

// src/pages/x.ts
import { APIRoute } from "astro"

export const GET: APIRoute = ctx => ctx.reroute("/y")

Middleware

import { defineMiddleware } from "astro:middleware"

export const onRequest = defineMiddleware((context, next) => {
    if (context.url.pathname === "/x") return context.reroute("/y")
    return next()
})

Background & Motivation

As astro grows in popularity, it is being used for more and more use cases. One of these is multitenant websites where a tenant may be given a subdomain. This proposal introduces a powerful primitive that allows astro users to build websites with complex routing requirements. Rather than a limitation, file-based routing is now a sensible default!

API

interface Astro {
    reroute(path: string | URL): Promise<Response>
}

The Reroute API (naming subject to bikeshedding) has prior art in these frameworks:

• Next.js: NextResponse.rewrite()
• others: TBD

Implicit rerouting

In addition to the introduction of an explicit API, this proposal aims to specify and discourage Astro's implicit rerouting behavior which has been involved in several high-priority issues (withastro/astro#7992, withastro/astro#8100, withastro/astro#8273, withastro/astro#8273).

---
- return new Response(null, { status: 404 })
+ return Astro.reroute("/404")
---

Framework-designed error pages

Importantly, the specification involves the addition of default, framework-designed 404 and 500 pages. These will be used when a user has not provided their own error pages. This follows users' expectation that middleware should run for non-existent routes (withastro/astro#7881, withastro/astro#8226).

The inclusion of default pages also brings Astro in line with other frameworks:

• Astro (current): blank page with a 404 status code which is not apparent to visitors.
• Fresh: framework-designed default 404 page.
• others: TBD

Recursive rerouting

If possible, core app's current behavior of sending http requests to its encapsulating server for 404 and 500 routes should be suppressed, if not removed. It is a ticking time bomb. The threat is that a user may encounter an "endless loop" bug in Astro code that turns into an overload or a runaway billing incident. Due to an unrelated issue, this has happened before on Vercel. While Vercel addressed the category of bugs on their platform, other deployment targets ae likely still subject to this issue, especially when self-hosted.

The number of .reroute() calls should be limited to 1 per visitor request. This decision aims to catch misuse where user code leads to (endlessly) recursive calls. The limit will be configurable as a global configuration.

export default defineConfig({
    maxRerouteDepth: 3
})

Goals

• Defuse.
• Make it unnecessary to rely on implicit routing rules
• Introduce framework-designed 404 and 500 pages
• Make middleware run for requests that don't match a route (with framework-designed error pages)
• Allow spread routes to opt-out of being matched

Non-Goals

• Reroute to an external service.

Detailed Design

Yet to be properly detailed, but the implementation will likely involve bringing RouteData or a closure to it from core App all the way to APIContext and AstroGlobal.

  App --- SSRPipeline ---                                                          --- Astro Global
                         \                                                        /
    ?? --- DevPipeline --->--- Base Pipeline--- Environment --- RenderContext ---<                   --- Middleware
                         /                                                        \                 /
 ?? --- BuildPipeline ---                                                          --- API Context <
                                                                                                    \
                                                                                                     --- Endpoint

See the exploration branch to explore ideas.

Testing Strategy

The feature could be adequately tested with a fixture-based suite.

Drawbacks

TBD

Alternatives

Astro.render(), context.render()

---
import FourOhFour from './404.astro';
return Astro.render(FourOhFour, { props })
---

// src/pages/api.ts
import Y from "./pages/y.astro"
import type { APIRoute } from "astro"

export const GET: APIRoute = ctx => ctx.render(Y, { props })

import Y from "./pages/y.astro"
import { defineMiddleware } from "astro:middleware"

export const onRequest = defineMiddleware((context, next) => {
    if (context.url.pathname === "/x") return context.render(Y, { props })
    return next()
})

For

• Matches with user expectations better.
• There is a clear path for functionPerRoute: imported routes get bundled with the functon.
• Adapter interoperability: since adapter involvement is not necessary, discrepancies between them is not a concern.

Against

• Maintenance effort: adapters that ran on non-node runtimes were the source of several bugs that took away from maintainer bandwidth. With the `edgeMiddleware` configuration, astro pages may be used on these runtimes again.
• In the frontmatter example, styles and scripts of the 404.astro page get included into the page.
• For highly dynamic routing, importing becomes unergonomic (click to open code example)

  import A from './a.astro'
  import B from './b.astro'
  import C from './c.astro'
  import D from './d.astro'
  import E from './e.astro'
  import F from './f.astro'
  import G from './g.astro'
  import H from './h.astro'
  
  export const GET = ctx => {
      switch (condition()) {
          case '/a': return ctx.render(A)
          case '/b': return ctx.render(B)
          case '/c': return ctx.render(C)
          case '/d': return ctx.render(D)
          case '/e': return ctx.render(E)
          case '/f': return ctx.render(F)
          case '/g': return ctx.render(G)
          case '/h': return ctx.render(H)
      }
  }

Harmony

How might this proposal interact with other in-progress proposals?

• Container API: render components in isolation #533 - The alternative API .render(AstroComponentFactory) shares use-cases and design considerations with this stage 2 RFC. In a way, the Container API is the environment-agnostic version of the alternative.
• i18n routing #701

Adoption strategy

Early stages; this is speculation.

While the explicit API can be enabled by default, the safety goal may require deprecating documented behavior. An opt-in configuration may be necessary for stability. Removing the self-calling behavior may need to wait until 4.0.

export default defineConfig({
    safeRouting: true
})

Unresolved Questions

• Astro-Adapter boundary: Some of the implicit routing changes would require participation from the adapters.

[matthewp]

I like this Arsh, good job! Thinking about it, one of the alternatives we've talked about is something like this:

---
import FourOhFour from './404.astro';

return Astro.render(FourOhFour, {
  // props here...
})
---

Curious to get a comparison between this and the reroute idea.

[lilnasy]

Initial thoughts: If we are thinking just about astro files, no major differences jump out to me. But it opens up to astro components being imported into middleware, and api routes. That may mean nothing but I would be biased against that pattern.

[lilnasy]

My current view is that Astro.render should be considered a part of the Container API.

While Astro.render can satisfy the use cases of the reroute API in certain cases, it would mean importing every astro page when rerouting logic is truly dynamic.

The container API can serve as the lower-level fallback API for the edge adapters where the platform does not provide a hook.

I advocate for continuing this proposal with the currently shown API.

[ematipico]

I like this proposal! Thank you for putting your thoughts on it, I don't have significant concerns.

One thing we should do when designing the APIs and the implementation, is to think about all the possible combinations of routes we could encounter.

---

Edge Middleware: By design, no single function would have access to all RouteData, which presents a challenge. How does Next.js deal with it?

I believe this must be done at the adapter level if the adapter can provide the functionalities. For example, Vercel provides a rewrite function from @vercel/edge package. Maybe in this case, the adapter should give Astro a reroute implementation. We could use the adapter feature for that.

Astro-Adapter boundary: Some of the implicit routing changes would require participation from the adapters.

Similar to my previous answer. We could expose adapter features to provide flags and functions.

[lilnasy]

Maybe in this case, the adapter should give Astro a reroute implementation.

If that's possible, then the code path for it is already clear. I will have to look into @vercel/edge to see if it can be plugged in there.

[kristoffermads]

As per request, I will chime in here :) First off, great stuff, this will get Astro a great bit further.

As a PHP programmer with a background in Laravel, I have always been a bit baffled with the acceptance of loss of control, when it comes to URLs, that we see in, for example Next or somewhat in Astro - which is nowhere near present in Laravel. In Laravel, all URLs are manually controlled, through a routing-scheme, which connect endpoints, to controllers.

With that in mind, I would like to take the oppotunity to take a step back, and view Astro a bit in that perspective - Laravel. Because for me, that would be the perfect level of control. (Actually, a lot of Laravel would be perfect for a lot of Javascript libraries/frameworks, but it seems for the moment, we are rediscovering :) )

I like to imagine the future of Astro, where the current file-routing exists only as an optional middleware. Where its the default - but not the only - routing middleware. Where the routing logic we know today, which fetches files from /pages, is just one of many possible routing logics.

Where if I wanted to, i could have three completely different endpoints, point to the exact same rendering-logic / astro component. Where I am not bound, by where my files are stored on a drive.

For example, I am currently working on a website where / (slash) of any URL, equals the frontpage, thread, forum or post, but where /page2, /page3 etc, equals the same as / (slash) but page 2 or 3 or etc of comments, threads, foras (etc). To achieve this in Astro, I would either have to:

1. Have two completely equal files where, when i edit one of them, i must remember to copy the content of that one to the other one, for example /pages/forum/index.astro and /pages/forum/page[num].astro . This is less hacky, but also a less cleaner dev-experience

2. have the /pages/forum/page[slug].astro file contain the logic, and have /page/forum/index.astro fetch the content from the "/forum/page1" url - but remember to reject all access to /forum/page1 from the outside - otherwise i would get duplicate content (SEO no-go). This feels alot hacky, but a more cleaner dev-experience.

As proposed by Arsh (and myself in an older thread on Discord #feedback-ideas https://discord.com/channels/830184174198718474/872579324446928896/1134522105543794819 ), there are other ways of solving this, which would have implications for other Astro-problems aswell, like the 404.

One way to solve this, would be to give access to "component rendering". If only we could return rendered Astro components in the frontmatter, alot could be possible. For example, as described above, we could have 404 without the extra fetch.

With the ability to render and return other components, we would also be able to do one simple /pages/[..slug].astro file and build a completely different routing system based on that file alone. It might not be pretty - but im sure if it was possible - that would be done.

But the best case scenario for me, would be a dedicated routing middleware, where the input is the request, and the output would be either "an astro component(aka:file) to render as per usual" or even better, the ability to return self-rendered components, directly in the routing middleware. Also, ofcourse, remember to be able to add on data already fetched in the middleware (so we can save those precious database-lookups)

Thanks for the proposal - hope to see this implemented in one way or the other :)

[lilnasy]

Framework-designed Error Pages - Prior Art

Astro

These are chrome's fallback pages as Astro renders nothing at all. Firefox and safari show a completely blank page.

Next.js

Nuxt

Remix

SvelteKit

All are running production builds with minimal changes to each framework's basic template. For consistency, the node adapter was added to the Astro project, and app.vue was removed from the Nuxt project.

[lilnasy]

Re-implemented as an adapter feature: https://github.com/lilnasy/astro/tree/proposal-reroute-new-impl.

Demo implementations:

• Node implementation
• Vercel implementation

Live demo with adapter: vercel({ functionPerRoute: true }): https://astro-reroute-pwv0wftcw-lilnasy.vercel.app/about.

Live demo behavior in brief:

• /about reroutes to /blog 50% of the time.

---
import Layout from '../layouts/BlogPost.astro';
if (Math.random() > 0.5) return Astro.reroute('/blog')
---

• /virtual:* reroutes to /*.

export const onRequest = (ctx, next) => {
    if (ctx.url.pathname.startsWith('/virtual:')) {
        return ctx.reroute(ctx.url.pathname.replace('/virtual:', '/'))
    }
    return next()
}

• /virtual:about will attempt to reroute to /blog but fail because rerouting multiple times is not allowed.

[mikestopcontinues]

This would be a meaningful improvement. However, I think there should be a default case to simply pass routing onto the next route that matches. This way, the user can continue leaning on the Astro router without very different data routes having to know where to route to next.

For example, take this route structure:

• /[product]/[feature]/[...slug]
• /[term]/[...slug]
• /[solution]/[...slug]
• /[...slug]

In the above situation, I would like to be able to call Astro.reroute() with no arguments, and let the router try the very next match. If the very last possible route calls Astro.reroute(), that's just a 404!

This would make it easy to run advanced matching rules within a route file, such as regex matches, without breaking the filesystem routing pattern.

[mikestopcontinues]

I would argue that fallthrough is the more Astro-like solution. By this, I don't mean it should be the only solution. You have some excellent use-cases for targeted routing above.

But consider a large codebase with only targeted rerouting. Each route will have to know (a) if the current path matches this route, and (b) which route Astro should try next. If you have five routes, this will mean A -> B -> C -> D -> E. To add a new route, you need to understand the un-typed logic spread across five other routes—and you need to know they all exist. At the same time, you're really only asking for a fallthrough. And the benefit of the fallthrough method is that each route only has to know (a) if the current path matches.

In a normal Astro codebase, it's already pretty common to see something like this:

const post = await getEntry('posts', slug);
if (!post) return new Response(null, {status: 404});

Fallthrough would simply augment this behavior. Rather than saying only "this route doesn't match," you'd also be saying "but some other route might:

const post = await getEntry('posts', slug);
if (!post) return Astro.reroute();

Fallthrough is really how Astro already works. This just gives the user more control over when fallthrough occurs.

---

For my particular case, regex would be a poor fit. I need to check multiple collections and taxonomies for a match to determine if the current route is a fit. E.G. "Is every piece of content in the route real and published?" If so, it's a match. (See my routes above for pretty good example.) This makes regex non-optimal for my use-case. It also makes injectRoute a decent hack, but not a perfect fit.

At the moment, I'm stuck having to use a single [...slug] route, then testing it against each potential match before rendering a particular layout. However, this means (a) I've reinvented a router, and (b) I need to lift all of the headers/redirect logic into the main file, because Astro seems to begin streaming before any nested layout component is rendered.

To refer back to the code sample, some of my routes would use a mix of 404s and reroutes, depending on if part of the path matched. But ultimately, I don't care which order these slug routes are tested. I just need to say "not this one."

[lilnasy]

Thanks for the detailed reply, it really helps.

I think the current proposed design works for use case, just from a different angle - you would use a middleware to attempt fallthrough.

export const onRequest = async (ctx, next) => {
    const originalResponse = await next()

    // routes intended only for rerouting
    const responseA = await ctx.reroute(`/_A?${pathParamsToSearchParams(ctx.request.url)}`)
    if (responseA .status !== 404)  return responseA 
    
    const responseB = await ctx.reroute(`/_B?${pathParamsToSearchParams(ctx.request.url)}`)
    if (responseB.status !== 404)  return responseB
    
    const responseC = await ctx.reroute(`/_C?${pathParamsToSearchParams(ctx.request.url)}`)
    if (responseC.status !== 404)  return responseC
}

Admittedly, this design is probably more manual than what I think you are hoping for. But I have seen other frameworks with a lot of implicit rules here, which is just more mental overhead. I would prefer to keep things explicit and not follow suit if it can be helped.

Regarding current fallthrough, you bring up a good point. Astro currently lets { code: 404 } responses fallthrough to src/pages/404.astro. Unfortunately, this often surprises users, and has edge cases around merging headers from the rwo responses. It also means users can't generate their own response body with a 404 status code - it will always be replaced. I plan to introduce an opt out with defineConfig({ implicitRerouting: false }) .

[mikestopcontinues]

I could definitely get by with the middleware. Thanks for pointing it out. I'm sorry I have to keep pushing on this... 😉

When I mentioned the existing Astro fallthrough, I mean that Astro currently sorts your routes, then tests each route pattern before trying the next. This pattern definitely isn't going anywhere. Everyone is familiar with it, as all routers work this way. I don't see fallthrough reroutes any differently. Really, you're explicitly saying reroute() to the next route in the stack. It won't happen otherwise. So this isn't concealing the routing logic, but rather leaning on it.

This method also avoids sometimes following the filesystem routing logic and sometimes following the middleware logic. You only need to keep one model in mind with fallthrough. To debug a problem, you only need to go to the route you expect should load and the route that actually loads. One of those two files will have the problem.

The middleware solution adds another dimension of routing logic—essential in some circumstances, but preferably avoided. After all, if people wanted to put their routing logic in middleware, we wouldn't have filesystem routing at all, but a classic code router.

[lilnasy]

This has come up multiple times since you brought it up, so you were clearly onto something! I have updated the goals to include it.

Astro.reroute isn't final yet, so it is possible for the API of this "fallback" feature to be something completely different.

What do you think about something like export function getDyanamicPaths(): string[] or export function matchDynamicPath(path: string): boolean?

[lilnasy]

As one example, Nuxt has a validate() function: https://nuxt.com/docs/getting-started/routing#route-validation.

[natemoo-re]

Very in favor of having an API like this!

Not to bikeshed on the naming, but I personally don't love reroute as a method name. I understand why you chose it, but I think rewrite might be more intuitive? That's what vercel.json, Next.js, and others seem to use.

[lilnasy]

Not set on the naming, but I intentionally went for something different from "rewrite".

Rewrites are implemented elsewhere in a way that the same API works for rewriting to an external service as well, the method essentially behaving like fetch(). My intention with using a different word was to avoid bringing in that connotation.

Not set on that behavior either, but I wanted it to be predictable how the request would be handled - always internally.

[mb21]

One use-case that seems possible with Astro.render but impossible with Astro.reroute, would be to actually render the error message on a 500 page, which can be very handy when users send you screenshots of your app. Or also customizing the 404 page on a case-by-case basis:

---
import NotFoundPage from './404.astro'
import ErrorPage from './5xx.astro'

try {
  const widget = await db.findWidget(Astro.params.id)
  if (!widget) {
    return Astro.render(
      NotFoundPage,
      { message: `Please see <a href='/widgets/'>all widgets</a>` }
    )
  }
} catch (err) {
  const status = err instanceof DBError
    ? 503 // temporarily unavailable, db may be overloaded
    : 500 // probably our code crashed somewhere
  const props = { err }
  return Astro.render(ErrorPage, props, { status })
}
---
<Layout>
  <Widget widget={widget} />
</Layout>

My feeling is that eventually we'll really need something like Astro.render or renderAstroComponentToString. Not sure whether there are also use-cases that are possible only with Astro.reroute and not with Astro.render, but if so, then we'd probably eventually need both.

[lilnasy]

In middleware, you could wrap next() in a try block and set a local with the error in the catch block before calling reroute. I haven't added this interaction to the proposal because it isn't at a stage where that level of detail is expected.

The downside with render in that example is that styles and scripts from 404.astro and 5xx.astro will always get included into the page. At least that is the way Astro is currently architecture - imported components automatically bring in js and css at build-time, whether or not they get used at render-time. There would need to be a way to opt of this behavior.

---
import NotFoundPage from "./404.astro" with { styles: false, scripts: false }
---

You do bring up a good point about render though. I think it's natural for Astro.render to exist after the container API is implemented.

[mb21]

Thanks for the reply!

The downside with render in that example is that styles and scripts from 404.astro and 5xx.astro will always get included into the page

ah, you mean into the normal page, even if no error was encountered? Now I get it. You may want to clarify that in the original post. And is that going to be solved by the container api?

[lilnasy]

Not necessarily, but it could.

The container API is currently intended for projects that are not built on Astro (the framework) that want to use Astro (the HTML templating language).

What happens when the API happens to be used in the framework has not been discussed.

[mb21]

Just a few more thoughts:

As astro grows in popularity, it is being used for more and more use cases. One of these is multitenant websites where a tenant may be given a subdomain.

Okay, but subdomain is actually not a problem at all? You just add a middleware that reads the tenant from request.host and puts it on locals, and then write the code to act accordingly? Or you mean if every tenant has a very different website? But then perhaps you should just create multiple Astro projects? Sorry if I'm dumb, but I would be interested to hear about the exact use-case you had in mind here.

This proposal introduces a powerful primitive that allows astro users to build websites with complex routing requirements. Rather than a limitation, file-based routing is now a sensible default!

That sounds exciting! One use-case we had for a past project that we considered migrating to Astro was that for SEO-reasons we wouldn't have been allowed to change the existing urls. There, /category would lead to a different kind of page than /slug-1234, which was natural to do in the legacy codebase, which used a regex-based router (think Ruby on Rails), but awkward with a file-based router. I totally can see how usig Astro.reroute in a middleware would solve this. But seems Astro.render would solve it just as well?

I also don't understand the following point:

[Against Astro.render:] In the frontmatter example, styles and scripts of the 404.astro page get included into the page.

When wouldn't you want the 404 page's styles to get included in the 404 page?

[mb21]

Okay, playing devil's advocate here (well, actually Astro.render's advocate).

For highly dynamic routing, importing becomes unergonomic (click to open code example)

Assuming Astro supports dynamic imports, that example could be solved like this?

export const GET = async ctx => {
    const moduleName = condition().toUpperCase()
    const Page = await import(moduleName)
    ctx.render(Page)
}

[lilnasy]

Or you mean if every tenant has a very different website?

Yes, each subdomain may need to be routed to a different set of asstro files.

But then perhaps you should just create multiple Astro projects?

Yeah, that is a valid approach as well. Whether or not it makes sense depends on how much is shared between the sites.

But seems Astro.render would solve it just as well?

Yes. Once implemented, either of the two APIs would solve the same problems.

When wouldn't you want the 404 page's styles to get included in the 404 page?

No, this has to do with the strategy Astro uses to decide which styles and scripts get included onto a rendered page. Because of streaming, we don't necessarily know which imported components are used at the time we send the <head>, so every imported component's assets get included even if it is rendered conditionally. I briefly mention one possible solution in a reply to your previous comment.

Okay, playing devil's advocate here (well, actually Astro.render's advocate).

Ha, go right ahead. It is all very welcome. Astro.render is a good API too, and - from what I've noticed - matches what users imagine the API for this feature should look like better than reroute.

My main hesitation with it is the higher implementation cost, and the fact that it probably depends on the container RFC being written and implemented first.

Assuming Astro supports dynamic imports, that example could be solved like this?

Unfortunately, it does not. Even when the container API is implemented, it will not look this simple because Vite needs to see an actual string inside the import call to transform it.

[cdtut]

To have full flexibility to support any use cases there should be option to have traditional routing in addition to or instead of file based routing.

File based routing has limitations. But with traditional routing you can use url to determine in fully custom way which actual component to render return Astro.render("Component.astro", context)) just like any SSR framework. (context would be used in component template.)

With such ability Astro can replace every SSR framework that exist.

Hono framework offer different routers https://hono.dev/concepts/routers for full speed and flexibility.

[tropperstyle]

I'd like to share a single page.astro for many routes, so I found this discussion while looking for more control over routing.

In my use-case, I do know the routes at build-time, so I thought injectRoute() would suffice.

Unfortunately, injected routes can only have one unique entrypoint.

I think my only option is to duplicate the file 8 times, or wait for a proposal like this to land.

[mb21]

my only option is to duplicate the file 8 times,

or move the file contents out to an astro component that you can use in those 8 files...

[tropperstyle]

Sure, but I use SSR and need to run code that may modify the response headers. This can only be done at the page level because of HTTP Streaming

[mb21]

Ah, right... you could still move that logic out to a helper function, of course. But It's true that it's not so nice to separate it from the component...

[ematipico]

Closing as accepted. #896