Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more real world test cases for more accurate results #6367

Closed
kartikk221 opened this issue May 17, 2023 · 1 comment
Closed

Add more real world test cases for more accurate results #6367

kartikk221 opened this issue May 17, 2023 · 1 comment

Comments

@kartikk221
Copy link
Contributor

kartikk221 commented May 17, 2023

Hello,

I maintain hyper-express and have recently been doing research to further optimize ahead of packages like sifrr and routejs-uwebsocket. The challenege I have run into is that these packages are not wholistically faster than hyper-express but rather are able to do away with certain operations at a sacrifice of common requirements for web / api development.

For example, the source code for hyper-express initial Request object creation is the following https://github.com/kartikk221/hyper-express/blob/9fce0cecc12c46a755d1f52272fd4209cbe44336/src/components/http/Request.js#L64-L87 and as you may observe hyper-express has to cache the request url, query, method (sometimes), headers (very expensive) and the path parameters (sometimes) which all become overhead but are a neccessity If the user ever works with asynchronous handling and decides to access any of the above properties throughout the lifecycle of a request. This is especially important because these properties are only available for the initial synchronous execution from uWebsockets.js hence they cannot be retrieved later on in the request lifecycle.

In contrast, both sifrr and routejs-uwebsocket are able to get away with not caching the above properties because they either don't support them or require you to declaratively retrieve them on the initial synchronous execution which makes these webservers not very developer friendly for projects given that they would be extremely barebones and essentially slightly expand on uWebsockets.js but still require you to call uWebsockets.js methods to do common tasks.

While this would be a huge undertaking but introducing the following additional tests would certainly be helpful overall to make the results more accurate for developers looking to choose the fastest and best developer experience webserver:

  • Body echo test where a POST request sends a string in the body and checks for the exact content in the echo
  • Both Path Parameters and Query Parameters are benchmarked. So something like /:id?value=something.
  • Headers echo test where some custom headers are sent and then echoed in the response body / checked.

Adding the above tests should not introduce any dependencies and should still remain true to the purpose of this benchmark which is the pure performance of the webserver rather than anything else like JSON parsing etc etc.

The above tests should require all webservers to actually retrieve almost all of the metadata of the request / response and thus better represent the real world performance for these webservers as then the URL parsing, body parsing and headers parsing are all equally tested on webservers.

@waghanza
Copy link
Collaborator

Hi @kartikk221,

There is 2 subjects you mention here.

First, is that some frameworks internals turn the figures here at their advantages.
Second, the scenarios here at not exactly reflecting at is a real-world workload.

The first, and the second are connected.

The idea / planning is to have more reliable scenarios facing real-world workloads, this is the next step after #3958.

Regards,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants