Redis / PostGreSQL Timeout in Heroku

[afzal-lakdawala]

Hello,
We are trying to implement Dashing-Rails for last 2-3 days. We are storing out datapoints in DB and then pulling it in jobs and passing it through send_server. The app completely crashes when two users (more than one) tries to access the dashing dashboard at the same time due to DB connection time out or connection refuse.

[gottfrois]

Hey,

That's interesting ! I'll try that myself and see what are the configurations required on Heroku for multiple connections. In the mean time you might want to read the following:

https://devcenter.heroku.com/articles/concurrency-and-database-connections

[gottfrois]

Hi,

I have build a new heroku application from scratch using dashing-rails 2.x.x and everything works fine. Please make sure to follow the wiki about how to run dashing-rails on heroku here

Please feel free to re-open the thread if you have any questions.

[ericboehs]

I haven't tested this app but looking at the code it looks like an identical problem I'm having with Heroku/Rails. In your config you set the db pool to 5. Rails reserves a database connection for each open request (even SSE), so as soon as you hit 5 concurrent users the site will stop working (these are people simply sitting there looking at the dashboard, even ones that may have just left the page open in the background). I'm trying to find a solution for this now. You could increase the pool size but Heroku limits you to 20 db connections on dev plans and 60 on their lowest production plan.

This may be fine for an app like dashing but is unacceptable for almost any production use case of SSE/EventSource on Rails. The real problem is Rails reserving a db connection for each request even if the request isn't connecting to the database (which is the case for dashing).

There's a whole other problem with the way dashing subscribes to redis. Most free redis add ons on Heroku limit you to 10 connections. Since (p)subscribe maintains an open connection, you have another bottleneck. Throwing money at it will fix it, but I'm working on another solution involving polling redis for changes (a very short 1 second poll so it still look instantaneous but it increases simultaneous connections by 100x+).

But that's an unrelated timeout issue, I'll continue working on the AR connection reserving issue and report back.

[ericboehs]

Another issue, which is probably causing this issue, is Rails has no way to know if the connection is still active. SSE is TCP based and clients send a FIN packet to terminate the connection but rails does nothing with this information. This is an open bug in rails.

I'm currently using the heartbeat workaround suggested by @njakobsen.

Unfortunately adding just that won't fix the issue. You'll need that fix, as well as preventing AR from creating multiple connections.

Research continues...

[gottfrois]

Amazing explanations thx ! Looking forward to hear more of your search on this, pretty interested ! Thanks man