benchmark critic

[vandergrafgenerator]

You just testing routers which does not represent any real use case.

For example the non blocking implementations are always on top with incredible scores but then you add unavoidable blocking code like http sessions or anything else an actual web app needs and the performance is downgraded to about the same as the average blocking implementations or even worse.

[waghanza]

You are right @vandergrafgenerator. The current implementation is not aligned with production usage.

This is a thing we are aware of. We plan to change this in #3108.

Any suggestion / idea is very welcome ❤️

[vandergrafgenerator]

@waghanza
Here are my suggestions:
First i do not know how you are running the benchmark tool but it should be run continuously for a few minutes because most app servers or whatever is the middleware if any are bugged causing severe performance degradation over a short period time when continuously hit. All implementations also should be fronted by an actual web server like nginx or apache, app servers or socket servers are not web servers even if they believe they are, you cannot run anything in production without an actual web server.

As for the app itself is the hardest part and i do not have any too specific suggestion. Must define a model that is the minimal of what most web apps do. For example i cannot imagine one app without server based sessions for all users, authentication checks(could be the session itself), some kind of dynamic html processing, DB queries, simply sending complete page html data of typical size to the socket(trust me, some implementations get a huge performance hit if they have to send more data than a simple hello world), Redis or Memcached often gets some usage but that may be too vague, hashing, sanitize and validate user input, accessing some outside service like a public or private API and at least tens of routes. That is what comes out of my head right now.

[waghanza]

it should be run continuously for a few minutes

exactly, the methodology is approximately the same https://www.techempower.com/benchmarks, but will be clarified when this project will be out of beta

All implementations also should be fronted by an actual web server like nginx or apache,

Using such app (nginx or apache) will add too much variation to the results @vaibhavmule

Must define a model that is the minimal of what most web apps do.

Agreed, I think to open a discussion when other stuff is done (see #3108 for details).

Feel free to give any valuable insight for this, worth to open a discussion imho

---

I close this issue, since this point is already on the roadmap