Remix HTTP Benchmarks : Remix App Server vs Vanilla Express server

[cliffordfajardo]

I've been doing some benchmarking at work recently and decided to test Remix with a vanilla ExpressJS app.
I'm seeing a vanilla expressJS web server handle 65% more requests than Remix currently.

Initially I was surprised, but perhaps this is to be expected since Remix is more than just a conventional HTTP server 🤔
Maybe this 65% difference is due to some overhead coming from Remix where it does some converting of the underlying server's request/response API into the Fetch API on the way in, and then adapting the Fetch Response coming from Remix into the server's response API? (as stated in the docs below):

While Remix runs on the server, it is not actually a server. It's just a handler that is given to an actual JavaScript server.
..
Adapters make them work for a specific server by converting the server's request/response API into the Fetch API on the way in, and then adapting the Fetch Response coming from Remix into the server's response API.
https://remix.run/docs/en/v1/pages/technical-explanation#http-handler-and-adapters

Github Repo: https://github.com/cliffordfajardo/remix-http-benchmarks

References

• Fetch API: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
• Remix Technical Explanation docs: https://remix.run/docs/en/v1/pages/technical-explanation#http-handler-and-adapters
• Benchmarking Github Repo: https://github.com/cliffordfajardo/remix-http-benchmarks

[cliffordfajardo]

I know you all are busy helping Remix become the best
Simply wanted to share this with ya'll and maybe get thoughts from community too.

Was just curious not looking to start a framework wars discussion / not my intent + perhaps get the performance nerds 🤓 curious about this so when there is a chance / time community may find ways to make Remix faster than it already is.

CC @ryanflorence @jacob-ebey 🙏

[sergiodxa]

Remember Remix has to do things like converting Node.js Request to Fetch Request objects, then grab your Fetch Response and convert it to Node.js Response objects.

Aside of that there's more work like matching the routes. Also did you tried in prod mode? Because in dev mode Remix will clear the require cache to ensure files are always the latest ones, so that will cause the server to take more time to process a request.

Finally, the less abstractions you add, the better results you will get, Express is a really tiny layer on top of Node.js built-in http server, while Remix is another layer on top of Express.

[cliffordfajardo]

Remember Remix has to do things like converting Node.js Request to Fetch Request objects, then grab your Fetch Response and convert it to Node.js Response objects.

glad to get confirmation and learn more about the abstractions that power Remix from a Remix power user like you

Also did you tried in prod mode? Because in dev mode Remix will clear the require cache to ensure files are always the latest ones, so that will cause the server to take more time to process a request.

Yeah I ran the benchmarks in prod mode npm start

Thanks @sergiodxa !

[jacob-ebey]

Could be a few things:

1. if you're running in development mode and not production, the cache is cleared every request and node has to eat the cost of loading modules
2. gzip seems to be disabled on the baseline and gzip isn't free, leading to the remix server doing extra work
3. the "morgan" logger is enabled by default in remix and logging isn't free
4. there is a cost in normalizing on the fetch API, but I wouldn't expect it to be this large
5. express.static is not the most efficient thing in the world. Removing the two from the starter almost doubles the requests per second for a basic JSON api resource route in remix

Locally after removing the logger and static file middleware, I went from 96k -> 144k requests/second. Still behind a bare-bones express API route doing 190k, but that's to be expected as it's doing ~1/2 the work. Adding express.statc to the baseline we go from 190k -> 155k requests/second.

The takeaway here though shouldn't be "remix is slow", it should be "understand the effects deps you take on have on performance, nothing is free".

In production if you need the perf, serve static assets from a CDN, not from express and remove logging or make it non-blocking.

[cliffordfajardo]

1. ✅ Ran in production mode by using npm start with these benchmarks

2. Regarding gzip

gzip seems to be disabled on the baseline and gzip isn't free, leading to the remix server doing extra work
Ahh it looks like gzip is one by default in the production Remix App Server, whereas in the vanilla express app it's not; that explains alot of the additional CPU bound work that's happening:

3. the "morgan" logger is enabled by default in remix and logging isn't free

4. Regarding express.static

• How would you recommend I disable the the morgan logger and gzip behavior for benchmarking purposes? I can probably update the benchmarks for patch-package

[jacob-ebey]

As Ryan said, eject to an express template and you'll have full control over your server. remix-serve is not extendable for a reason and is meant as a beginners production ready server with zero config for those uncomfortable setting up their own.

[ryanflorence]

Neat! Keep in mind the "Remix App Server" is not "Remix". It's a minimal production express server to get folks unfamiliar with creating servers up and running quickly.

So, I suspect what you're mostly comparing is the cost of the express middleware in the Remix App Server, but not Remix.

To actually measure the cost of Remix vs. bare express handlers, I'd use the @remix-run/express adapter directly, not remix-serve. You can bootstrap one with npx create-remix@latest, choose "just the basics" and then choose "Express". Then delete whatever stuff is in there that's not in your express server (like morgan, static files, etc). Also, remove the <Scripts/> so that the browser isn't wasting time on static assets that your bare bones express app doesn't have (which also allows you to remove the static assets handler).

And of course, make sure you're running a production build.

[ryanflorence]

Also note you're not sending HTML in your express server, you should probably at least render similar HTML somehow, or not render anything in Remix, too.

[ryanflorence]

Oops, one more thing: if you don't use React to render that HTML in your express server, you'll likely be comparing React vs. string concatenation rather than Remix.

So either stop using React in your entry.server.tsx and just concat strings, or use ReactDOMServer.renderToString in your express server to actually see the cost of Remix.

In other words: good benchmark comparisons are hard to do well 😅

[cliffordfajardo]

In other words: good benchmark comparisons are hard to do well 😅

💯

[jgb-solutions]

@ryanflorence @jacob-ebey guys it looks like having a comparison article (à la Next.js vs Remix) for the adapters would be great. I go with the default for certain platforms without no big idea about their advantages or inconveniences, especially for Remix server vs Express, etc. Otherwise from my short experience with Remix over the last month or so, I can actually say that sites/apps built with Remix are faster than CRA or Next.js, assuming that your backend is fast enough.

[jgb-solutions]

@ryanflorence sounds fair. By the way does Remix App Server adds long cache headers to the static files by default? By playing with page headers to get maximum performance on a CDN, this was not my experience. Either I did something wrong or it doesn't handle that. Or should I switch to the express adapter to have control over that? From the above conversation, it sounds like that's what I should do to squeeze more juice out of the framework. Though I don't really need it for now but I'm crazy about performance as well.

[ryanflorence]

It does immutable for the assets Remix creates in public/build/*, and one hour for anything else in /public/*. For all other requests it's up to you.

If you have a CDN and you don't see the right cache-control, you may have settings on your CDN that override anything the origin server is sending.

[jgb-solutions]

@ryanflorence thank you! Yeah I saw the one hour. I just want it to be longer. I don't remember having seen anything related to that in the docs on how to update that value. That way the browser will cache them for longer duration. Vercel and Cloudflare do cache static assets by default though. So that's a no brainer.

[sergiodxa]

@jgb-solutions the only way yo change the 1 hour cache for non built assets is to switch to Express instead of using Remix App Server

[jgb-solutions]

@sergiodxa that's what I concluded. Thank you for confirming!

[cliffordfajardo]

Update:

My original question is resolved ✅ . Thanks Remix team!

Want to confirm that following what @jacob-ebey & @ryanflorence recommendations about using the express template which exposes the underlying server & disabling the logging & compression got my performance up 65% to match the benchmarks of the non-remix express server I initially mentioned in the description of the first post I made.

More Benchmarking related stuff

⬆️ I updated the examples in my Remix benchmarking Repo;
I added Remix FastifyJS example thanks to @mcansh @mcansh/remix-fastify repo which is awesome - thanks Logan!

Remix-Fastify was averaging 13% better throughput than Remix-Expresss
I'll add more examples using the Fastify's recommended pino-logger which should yield better performance than morgan logger used in Express. More benchmark variations to come...with more documentation and details

FYI: I'll stop post benchmarking updates here, I'll keep those in the benchmarking repo:
https://github.com/cliffordfajardo/remix-http-benchmarks

[jgb-solutions]

@cliffordfajardo nice! Remember to add a description to that repo.