High failure rate versus Puma on a small Sinatra app #277
Description
I don't know how useful this data point is to you, but I'll drop it here and you can feel free to ignore it.
At my workplace (proprietary, so unfortunately I can't share the full source) we deploy an app with the following characteristics:
- Very small and lightweight app; ~2 meaningful endpoints
- Runs on the Sinatra framework and Puma server; vanilla MRI runtime, Ruby 3.3.6
- Default configuration for Puma (maximum of 5 threads)
- Deployed on a single AWS EC2 instance (a
t3.medium) - Each request triggers one blocking upstream request to an AWS service
- About 50% of the processing time for each request is spent waiting for those upstream requests to finish
As an experiment, I tried switching from Puma to Falcon, expecting that with non-blocking fibers we'd get a significant increase in throughput. To prove this out, I wrote a k6 load test that ramps up linearly from 0 to 100 simultaneous VUs over 10 seconds, holds steady at 100 VUs for 5 seconds, and then ramps back down to 0 users over 5 seconds.
As a baseline, with our regular setup we were topping out at 65 requests per second with a 0.36% error rate (HTTP responses with a status code other than 200, or an empty/incorrect response body).
- For a naive configuration of Falcon (no preloading), throughput increased to 75 requests per second, but error rate increased to 20%.
- With preloading the
app.rbfile, throughput went up to 90 rps, but error rate increased to 40%. - With some more careful preloading, including app configuration (to avoid any simultaneous file handle issues) performance was nearly identical to the naive configuration.
I don't have great visibility into what was going wrong on failed requests, but I wonder if Falcon doesn't have the same request queuing functionality as Puma or something.
Let me know if you have any followup questions.