I added Agoo to the TFB

[GertThiel]

https://github.com/GertThiel/FrameworkBenchmarks/tree/master/frameworks/Ruby/agoo

Maybe you want to have a look, maybe want to optimize?

[ohler55]

The code looks pretty reasonable. The only thing I would change might be the number of workers. That is something you would have to experiment with though. I suspect the database will be the most significant bottleneck since it looked like the queries are synchronous. Spreading them out over as many separate real threads will probably give the best performance. I wonder if the connection pool make a significant difference or if more workers would give better results.

Nice charts by the way.

[GertThiel]

The charts aren't mine. I only added Agoo to the TechEmpower Framework Benchmark (TFB): TechEmpower/FrameworkBenchmarks#4709

I suspect the database will be the most significant bottleneck since it looked like the queries are synchronous. Spreading them out over as many separate real threads will probably give the best performance.

Do you have sample code how to accomplish that?

[ohler55]

Just change the worker_count in app.rb and maybe the size of the connection pool. If it is easy for you to run the test try multiplying the worker_count by some scaling factor and see what yields the best results. Using a machine close to the one they do the benchmark runs on would be best of course.

[GertThiel]

The TFB runs the actual benchmark in Docker containers. worker_count is derived from nproc (see here). The pool is set as big as set by the competing frameworks( everyone uses about 250).

The machines which will run the benchmarks is considerably different from mine. So nproc seems to be a good strategy to configure worker_count except you think a multiplicator would be better (Two workers per processor core?).

[ohler55]

It might be better to have a multiplier on worker_count since most of the time will be spent waiting for the database.

I'm not sure how the connection pool works but if it establishes 250 connections right away that could be a problem if mongo is not configured for for a large number of connections. Let me explain a little.

The worker_count is used to create that number of separate Ruby processes. They show up as separate entries when running top. Each one is running app.rb so with the pool at 250 connections and lets say 12 CPUs thats a lot of connections. Looking at it still further we know that Ruby uses a global lock for each app so the app is locked until the database returns. Depending on the database driver it may allow other Ruby threads to run in which case increasing the Agoo thread_count could help. Having connections in the pool that are not used will only slow down the server as those connections still need to be checked and in MongoDB's case there is a new thread created for each connection. So I would suggest the pool size should be the same as the thread_count.

So I threw in another variable, didn't I. Not knowing how the drivers are implemented makes it tough to give specific values for the worker_count and thread_count but they are configurable so some trial runs with different values should allow the best values to be determined.

[ohler55]

What ever happened with this? I did not see Agoo on the results page.

[ohler55]

No response. Closing.

[ElMassimo]

What ever happened with this? I did not see Agoo on the results page.

@ohler55 In case you never found it, it was added in TechEmpower/FrameworkBenchmarks#4709.

Have in mind the implementation is flagged as Stripped, which means it's not shown by default, and needs to be manually enabled in the filters:

[ohler55]

I failed to find the implementation approach option for Stripped in the filters. All that appears is Realistic. I would love to see the results and would be glad to check the configuration to see if it is optimum. I would also like to get Agoo so it shows up with Realistic if you can point me at what to takes to earn that label.

[ElMassimo]

From the general requirements in the project:

This project intends to measure performance of production-grade deployments of real-world web application frameworks and platforms. All implementations are expected (but not required) to be based on robust implementations of the HTTP protocol. Implementations that are not based on a realistic HTTP implementation will be marked as Stripped. Rather than stipulate a rigid specification of HTTP requirements, we defer to common sense: the implementation should provide a viable subset of HTTP to reasonably unbiased outsiders familiar with HTTP.

Not familiar with how the decision is made, in this case it seems it was added as Stripped by @GertThiel.

Perhaps you can make a pull request with changes to the sample app to flag it as Realistic instead.

[ohler55]

Thanks. I'll do that and see what the response is.