Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable threadsafe! by default #6685

Closed
wants to merge 1 commit into from

Conversation

@tarcieri
Copy link
Contributor

commented Jun 8, 2012

No one uses thread-safe mode because it's disabled by default
This makes thread-safe mode configuration over convention

If thread-safe mode were the happy path, more people would use it

No one uses thread-safe mode because it's disabled by default
This makes thread-safe mode configuration over convention

If thread-safe mode were the happy path, more people would use it
@joevandyk

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2012

+1

1 similar comment
@tenderlove

This comment has been minimized.

Copy link
Member

commented Jun 8, 2012

👍

@tenderlove

This comment has been minimized.

Copy link
Member

commented Jun 8, 2012

TBH, I don't know why we have it disabled by default. Anyone know why?

@bts

This comment has been minimized.

Copy link

commented Jun 8, 2012

👍

1 similar comment
@cfcosta

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2012

+1

@drogus

This comment has been minimized.

Copy link
Member

commented Jun 8, 2012

@tenderlove iirc it's disabled, because at the time it was added, much more gems and applications were not thread safe

I would also turn it on by default.

@collin

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2012

In 3.2.x I've experienced that this can bite you due to differences in the way autoload works in development/production.

When I switched threadsafe! = true some files that looked like this:

class Whatever::Something
end

Worked in development, but the app bit the dust in production because of the implicit Whatever module. If I understand correctly, autoload is not threadsafe. Is autoload going to be around in Rails 4 development mode?

If so, can anything be done about identifying this kind of error short of running the app in production mode before deploying? Or perhaps get the implicit module behavior working when autoloading is disabled?

edit: still 👍

@nahi

This comment has been minimized.

Copy link

commented Jun 8, 2012

I heard that Rails depends on atomicity of Hash#[]= for caching, but it's not true for JRuby. @tarcieri should know about it better though :)

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 8, 2012

@nahi where specifically? ActiveSupport::Cache::MemoryStore, for example, synchronizes with a Monitor:

https://github.com/rails/rails/blob/master/activesupport/lib/active_support/cache/memory_store.rb#L118

@thedarkone

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2012

@tarcieri here for example: resolver.rb#L35. See #6425 and other related tickets.

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 8, 2012

@thedarkone @nahi aah, I think that might work "by accident" on JRuby now but should definitely get fixed upstream. @headius would be the one to ask

@tammersaleh

This comment has been minimized.

Copy link

commented Jun 8, 2012

👍 turning this on by default forces the discovery of these issues.

@nahi

This comment has been minimized.

Copy link

commented Jun 9, 2012

Yes @thedarkone, you're the person I thought of :-)

@fxn

This comment has been minimized.

Copy link
Member

commented Jun 9, 2012

To choose a default you need to ask yourself what is useful to most end-users. Well, the fact is that most projects are deployed using Passenger and Unicorn. Multi-process is dominant. Therefore, the default should be off in my view.

@anildigital

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2012

👍 must go in.

@stevenh512

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2012

Is there still an issue with certain rake tasks (specifically db:seed and any other task that needs access to models) not working when threadsafe is enabled?

@mcmire

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2012

+1 on the idea. I don't think it'll be as easy as flipping a switch, but I think it's worth it we really start thinking about this and moving in this direction as a community. There's no reason why Rails shouldn't promote a multithreaded mindset by now, it seems like the next logical step after dropping Ruby 1.8. Also, yes it's true we use Passenger and Unicorn nowadays, and they're both great and I don't expect them to go away any time soon. But if I could use just one process in exchange for having to be more conscious about thread safety, that'd be great. (Seems crazy in Ruby-land but I think we've just gotten used to the status quo.)

@headius

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

So here's my question for anyone opposed...are you advocating that the default runtime mode for Rails be non-threadsafe? Really?

Threadsafe being on by default will mean bugs get caught sooner, performances issues from eagerly (or safely) loading libraries must be addressed, and there's no surprises when someone runs threadsafe on JRuby or Rubinius. There are now more implementations with parallel threads than there are GIL implementations. Perhaps it's time for thread safety to become the norm for Rails and all of Rubydom.

@stevenh512

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

@headius Just for the record (since I commented here), I'm +1 on this, I was just curious if the issue with rake tasks that need access to models still existed. In the little bit of testing I've done in the past couple days it doesn't seem to be an issue anymore. :)

@headius

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

@stevenh512 I'm not familiar with that issue, so I'll let others answer...

@stevenh512

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

In 3.0 (and if I recall correctly, earlier 3.1 releases) rake db:seed (or any rake task accessing models) with threadsafe enabled would raise an "undefined constant" error. Slightly annoying issue with a simple workaround, really.. but I can't reproduce it in 3.2.5 or edge Rails with threadsafe enabled.

@josevalim

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

Everyone taking part in this discussion and said 👍, can you please turn config.threadsafe! in development today in your apps and report back how it goes?

I agree it is important to run it in development to catch bugs but we need to do our due diligence before we merge this patch and before we ship Rails 4. I would rather handle 80% of the bugs now than after Rails 4 just comes out.

@josevalim

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

About resolver, we are aware that it is not threadsafe but since it is a cache, it should not be a problem if we have concurrent read/writes (i.e. we don't need to put a lock around the whole template lookup, just around the data structure access), except that JRuby does raise an error when it happens. The solutions provided so far moves all the logic into the resolver, while I think it should be handled by the underlying data structure.

EDIT: removed possible solutions because they were redundant

@tenderlove

This comment has been minimized.

Copy link
Member

commented Jun 11, 2012

@nahi if we are relying on atomicity of Hash#[]= (or any other non-threadsafe data structure), I consider it a bug.

I think the threadsafe! flag is misleading. When you have threadsafe disabled, you're saying "I'm OK with running a non-deterministic system". Most people are not OK with running a non-deterministic system, they just don't know it will become non-deterministic (even in MRI) when threads are introduced. We cannot restrict users or libraries from using threads, and requiring people to a) track down that some library (or some user code) is using threads, then b) flip a magic switch that makes everything OK again seems like an unacceptable default.

My true opinion is that Rails should just always be threadsafe and the config flag is removed. But defaulting to threadsafe in the config settings seems like the most conservative approach for Rails 4.

  • If code is slow, let's make it fast.
  • If code is not threadsafe, let's make it threadsafe.

We have to do those things anyway, let's not do it half way.

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 11, 2012

Regarding threadsafe! mode in development, you are unlikely to see thread safety bugs in development because there's no contention, however you are likely to see autoload bugs which won't happen in production because production uses eager loading. Just my 2¢.

@josevalim

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

@tenderlove Agreed. I just wish we start doing our due diligence now, possibly backporting the latest thread safety fixes to 3-2-stable so we can try it out now. Based on my previous experience with config.threadsafe!, there were bugs related to autoload even in production mode where we eager load most of the stuff. Development mode uses another strategy for code loading, which may have even more thread-related bugs (I have no idea what will happen if two threads try to autoload the same constant at the same time via AS::Dependencies). Besides autoload I am also aware of issues in the resolver and possibly in the connection pool.

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 11, 2012

@josevalim for what it's worth, I've been somewhat interested in writing a lock-free cache that uses ConcurrentHashMap on JRuby and the Atomic gem elsewhere. I'd be shooting for something like the ActiveSupport::Cache::Store API

@josevalim

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

@tarcieri Seems reasonable. The thing is that the resolver currently uses a nested hash structure which I believe would be expensive expensive with ConcurrentHashMap. I think it would be better to use an array as a key. So basically different Ruby implementations will require different storing strategies, so an abstraction may not fit forcing us to solve the problem inside Rails itself (which is fine, imo).

@josevalim

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2012

@tarcieri are you sure there is no contention? I believe it is common to have an application that, while rendering, does a another request to load a json or an inline image file from the rails app. They could easily try to load the same constant in such cases, c/d?

@agibralter

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

When you do rails g controller admin/foos it puts

class Admin::FoosController < ApplicationController
end

in app/controllers/admin/foos_controller.rb

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

Are you stating that this works when threadsafe! is disabled but doesn't work if it is enabled?

@agibralter

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

Ah sorry -- it used to be the case that threadsafe caused trouble for implicit modules. Seems to work now.

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

@tarcieri that's exactly my point, eager loading would benefit users of CoW GC, and it would cancel out the benefit of using multi-threaded envs for less memory consumption. Even better, forking multi-process have true concurrency utilizing multi-cores on MRIs with GIL. My point is that, by the time truly CoW-friendly GC is available on MRI, writing thread-safe code - again, I'd strees that's for request concurrency - would be less beneficial, just adding code complexity.

But practically speaking, I wouldn't hope too much on CoW-friendly GC to work 100% as advertised, so I'd imagine myself using Puma for some projects, Unicorn for the other, and of course Celluloid for some. :)

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 21, 2012

eager loading would benefit users of CoW GC, and it would cancel out the benefit of using multi-threaded envs for less memory consumption [...] writing thread-safe code - again, I'd strees that's for request concurrency - would be less beneficial, just adding code complexity.

Please see my comments about how a lack of thread safety increases code complexity:

#6685 (comment)

I am speaking from the experience of how I have seen systems which were already CoW-friendly (i.e. REE w\ eager loading) handle concurrency. There is still the issue that requests that block for long periods of time can easily take down an entire site. If you receive N of these requests, your site will be completely blocked and unable to handle any traffic.

As a result, people attempt to keep render times short by placing long running requests into background job queues and polling for the result if desired. This adds a ton of incidental complexity when synchronous requests would otherwise suffice.

In the days before Rails was thread safe, I used to use a Merb application to handle long-running requests like this (e.g. ones that proxy to a slow external service or make calls to several services) Threads provide a lightweight way for a single VM to service several requests. CoW helps, but a VM is still substantially heavier than a pthread.

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

@tarcieri I don't get your argument. A similar problem would happen in a theaded environment as you'll also have a thread pool limit. The difference is that the threaded approach will consume less memory.

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

@tarcieri If a CoW-friendly env doesn't offer enough concurrency, so doesn't a multi-thread env. They are in the same order of magnitude of concurrency, like 100s per server at best. Definitely not in the order of 10,000s per server that evented model would offer.

I think we should separate out the background job sort of discussion here, that's orthogonal IMO.

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 21, 2012

@rosenfeld It's not just about memory usage. By using fewer processes with fewer heaps that the GC can be more efficient about managing, you will reduce CPU utilization as well.

@kenn are you seriously arguing that a process is the same weight as a pthread? That is not true whatsoever. I don't know how many workers you are running relative to your number of CPUs, but we can't run 100 processes per server with REE, even on servers with 16 CPU cores. Beyond just memory usage, CPU utilization (largely due to the garbage collector) is too high, and we see higher throughput by reducing the number of processes.

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

@tarcieri I don't think any GC optimization would be significative here.

Furthermore if I recall correctly, threads are implemented as processes on Linux

Also, I've made some benchmarks yesterday and using threads (pool limit of 50) and 50 processes served performed about the same with a fairly simple new Rails app. I've run something like "ab -n 150 -c 50 http://localhost:3000/test/index". The difference is that the multi-process approach required almost 1GB for a fresh minimal Rails app.

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 21, 2012

@rosenfeld "Furthermore if I recall correctly, threads are implemented as processes on Linux"

Even if this were true, which I can assure you it is not, "process" in this context equates to an entire Ruby virtual machine as compared to a Ruby Thread. I can certainly assure you that the costs of the former are significantly higher on all fronts, even with CoW+forking, than the latter.

Add a "sleep 5" into your controller to simulate access to a slow API and you will see a significant difference. Or better yet, just read @tenderlove's blog post:

http://tenderlovemaking.com/2012/06/18/removing-config-threadsafe.html

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

@rosenfeld on Linux, creating a thread eventually calls clone() and it internally calls do_fork(). Also fork() calls clone() and do_fork(). So yes that's true.

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 21, 2012

By definition, processes do not share memory (barring certain SysV oddities) and threads do. All that said, it's irrelevant to the discussion, where "process" equates to "Ruby VM", and "thread" equates to "Ruby Thread"

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2012

@tarcieri with that simplistic statement (threads share memory while processes do not), you are only talking about visibility of shared memory, but it could actually be the same physical memory, until it's mutated - that's indeed the effect of CoW.

But I agree with you that in the real world, CoW isn't nearly as effective as threads. CLONE_VM is the most significant option to clone() call, that is. I'm just pointing out that multi-thread is, at least in theory, very close to multi-process. And not worrying about thread-safety, with transparently shared memory, is a huge win for developers.

I've +1'd to your suggestion anyway, so...

@thedarkone

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

@kenn @rosenfeld The problem with CoW friendly processes is that they impede the most efficient GC algorithms (moving/generational GCs). It also quite difficult to CoW-friendly share the JIT-ed Ruby code.

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

@thedarkone Exactly, deploying threaded application is a way simpler approach and that is why only Ruby frameworks have largely adopted multi-process deployment as far as I can tell. Most stablished servers will spawn (or delegate) new threads on new requests, not new processes. But it seems that there is an historical reason for that since MRI always had bad threading support, specially before 1.9. But now that Rails is no longer targetting 1.8 I guess it should be time for changing the mentality.

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

Also, I can't really understand why people find multi-threading programming for web applications so complicated. Unless you do a concious choice of sharing state among requests this should never happen. And if you chose to do it then it is likely that you also know the implications and will put mutexes around the shared object which is also pretty simple. Web applications usually are much simpler than most other threaded applications that can require more effort for doing proper threaded programming. And even in those cases, it is just another discipline to learn. Threaded-programming is not that hard. People over-estime it. It is just that some people in the Ruby community are not used to multi-thread programming yet. But they are smart and should learn about it very quickly if they don't already know how to write multi-threaded safe code.

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

@thedarkone good point. there's always more complicated trade-offs in details, that's why I don't bet all my money on either threads or CoW. I found in this conversation that threads are overrated (even without GIL like JRuby) so I took the stance for promoting CoW a little bit.

But honestly, switching logical address space is costly (switch_mm() does nothing with threads) if you have 100s of them. Also as @tarcieri pointed out, GC overhead would be significant, as they run per process.

That said, I don't see any realistic rationale for threaded concurrency yet.

For instance, we need N+ processes for N cores anyway, assuming unfortunately we'll continue to have GIL. GIL won't go away anytime soon. And by saying N+, I mean up to N*2, depending on I/O ratio in our app. We could ask New Relic for real data but I believe ruby/db breakdown in time spent would be something around 2:1 in a typical Rails app. If your database is slower than this, adding ruby concurrency won't help anyway, it just makes the situation worse so fix your query. If we assume ruby:db = 2:1, we only need N * 1.5 processes.

Guess what I'm saying? With this typical breakdown scenario, we need at least N worker processes, whichever you choose Puma or Unicorn, to utilize CPU cores, and to save iowait, we only need 50% more concurrency. 2 threads per process is more than enough. Why not just add 50% more processes instead? You don't need 100s of concurrency for 16 CPUs. You need about 24 processes. That's roughly the range we're talking about, and in this range, 24 threads or 24 processes shouldn't matter much, with CPU affinity of modern kernels.

If you have a quad-core processor, and a process size is 60MB, you need 60MB * 4-cores * 1.5 = 360MB RAM (not considering CoW, even). And that's our real setup on Linode 512 (many of them btw), the cheapest plan on one of the most popular VPS.

We could argue that iowait is not just database, we have external APIs like twitter, facebook, etc. and that is a valid argument. But things can go crazy, yesterday Twitter went down for 2 hours, and if an app depends and blocks on Twitter, it will go down with Twitter anyway, no matter how many concurrency it had. If we don't want to let the entire site down, we need to have asymmetric control to external dependency (segment workers that do/don't handle external APIs), rather than having more concurrency. Or if you have 10,000s of external dependency like web crawlers, use an evented server. From this perspective, threaded concurrency makes very little difference from multi-processes in practice.

And I have to disagree with @rosenfeld on overestimated thread-safety. It's not that hard to write, if you are aware upfront, but it's extremely hard to track down the bug once it's slipped and deployed. You can never find the bug on development environment, and it's hard to write sane tests for them. How many people here have actually written tests for race conditions? If you have, you know what I mean.

So, not worrying about thread safety (both GIL and single concurrency per process) has been a big "feature". It's the best default, ever. And now we shed lights on another cool option.

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

After reading @tenderlove's post once again, I think that allow_concurrency is the one which should be removed completely. That's the responsibility of servers, not Rails. So removing allow_concurrency (Rack::Lock, that is) from threadsafe! bundle seems sufficient to me, no?

preload_frameworks, cache_classes and dependency_loading are all useful, so we could continue to bundle those three with threadsafe!. And threadsafe! should be enabled by default for production, as this ticket suggests.

@bts

This comment has been minimized.

Copy link

commented Jun 22, 2012

@kenn "For instance, we need N+ processes for N cores anyway, assuming unfortunately we'll continue to have GIL. GIL won't go away anytime soon."

There's no GIL in JRuby and it will soon be gone in Rubinius. I don't think we should hold Rails back because YARV can't handle threaded concurrency yet.

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

@bts sure, but what for? In my story, I said 6 processes take only 360MB of RAM. A single JVM with true threaded concurrency could use even less memory while fully utilizing CPU cores. Nice, but isn't 360MB small enough already?

In the future, we'll have much more cores, like 128 in a single box, and that's when no-GIL threaded concurrency will be truly useful, in a practical sense. Will that future come next year? No. What we've been seeing instead is that servers are going all cloud and those many cores aren't fully allocated to a single Linux VM. RAM is cheaper than CPU.

With a cloud with extremely poor CPUs the situation gets to the extreme. I've seen many setups on EC2 that there are plenty of free memory while CPU cores are pegged and load average is high.

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

@kenn Exactly. You're able to serve 6 concurrent requests with 360MB while you could serve about a hundred concurrent requests depending on what your application does in a threaded environment running on YARD.

I wouldn't be surprised if a threaded Redmine was able to serve bugs.ruby-lang.org in a 256MB VPS, but I don't thik it would suffice in a multi-process environment.

@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

@rosenfeld we're talking about throughput, right? If average response time is 50ms, a single concurrency could serve 20 requests per second. If you have 6, 120 requests per second. "A hundred concurrent requests" means a hundred concurrent query to the database (not considering connection pool).

A connection in the database is much more serious than ruby concurrency. It takes 8MB stack space + many other per-connection buffers for each. Too many Ruby concurrency than necessary is harmful, that's my point. In my story, 6 concurrency because 6 is optimal. Beyond 6 is too many.

@rosenfeld

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

Some requests could be consuming third-party APIs like Twitter, FB or GitHub instead of relying on a connection from database pool. Also, non RDBMS have a different memory allocation scheme.

It would be great if we could have a low-level control about our threads and/or processes. For instance, use rules like:

  • All database-dependent requests should share those 6 threads/processes.
  • All long-running report requests should share another 3 threads/processes.
  • Third-party API consumers should use another pool of threads/processes.
  • ...

That would be really awesome, but I guess currently we'd need something like a load balancing proxy for distributing requests like this... :( And that means that different processes would be required which prevent one from sharing in-memory state among those processes unless they use something like memcached.

@tarcieri

This comment has been minimized.

Copy link
Contributor Author

commented Jun 22, 2012

@kenn "If average response time is 50ms..."

I don't know what Rails apps you work on, but I have never worked on any site where the mean response time was 50ms. It has typically been much higher than that.

@rosenfeld "Some requests could be consuming third-party APIs like Twitter, FB or GitHub"

Want an example of APIs that are extremely important but notoriously slow? Payment gateways. Imagine it takes 5 seconds to process a payment (this is not unrealistic). Imagine you have 16 Unicorn workers on a given server (let's say 4 cores). This means with only 16 payment processing requests you have completely exhausted the capacity of this server, which is going to be sitting there idle while the payment gateway continues processing those requests.

Now let's say we use 10 threads per worker instead, and one worker per CPU instead of 4. We can now process 40 payments requests simultaneously on this same server.

These are not unrealistic numbers in either respect. Threads make concurrency cheaper.

@kenn "A connection in the database is much more serious than ruby concurrency."

I work on a very, very large Rails web site. Our main bottleneck right now is the CPU time spent in the garbage collector. Trying to eke out additional concurrency via REE processes (even with CoW) is not possible because we do not have the CPU capacity available thanks to GC time.

Database connection concurrency is a comparatively minor issue.

@tenderlove

This comment has been minimized.

Copy link
Member

commented Jun 22, 2012

Hey folks, I'm closing this pull request. We are going to remove the lock and enable eager loading for production, but not using this patch. I am personally in favor of @fxn's suggestions on adding a boot_strategy config, but I don't think this pull request is the right forum for that discussion.

@tenderlove tenderlove closed this Jun 22, 2012
@kenn

This comment has been minimized.

Copy link
Contributor

commented Jun 22, 2012

Some requests could be consuming third-party APIs like Twitter, FB or GitHub instead of relying on a connection from database pool. Also, non RDBMS have a different memory allocation scheme.

@rosenfeld At least MongoDB has the same memory allocation scheme, though the default stack size has been reduced to 1MB since 2.0.

I don't know what Rails apps you work on, but I have never worked on any site where the mean response time was 50ms.

@tarcieri probably because your app renders views. I find rendering views is the most CPU intensive part in Rails, but my app is mostly a bunch of APIs that feed JSON to iOS/Android devices. (and we're fanatical about keeping the json size as small as possible). In our case, response time ranges from 20ms to 200ms, average being just above 50ms. We off-load the rendering CPU cost to the client devices.

Or, if you're on EC2, it's because EC2 is extremely slow. :) Typically 2x to 3x slower than VPS or self hosted.

Payment gateways.

Ah, that is a legit and great example, in that case I would choose Puma over Unicorn. :) It makes sense to use threaded server for always slow external services. Maybe I'm too spoiled by Stripe and Apple's reasonably fast In-App Purchase verification server.

Our main bottleneck right now is the garbage collector.

I also work on a significantly large Rails app with 15 million users on mobile devices, and there I have experienced many kinds of problems, but never seen GC to become the primary source of bottleneck. Again, probably because we don't render large HTML views, mostly JSON.

@jrochkind

This comment has been minimized.

Copy link
Contributor

commented Nov 4, 2012

I am actually curious about how threadsafe mode might affect MRI deployments, actually. In theory, you could improve throughput of a single MRI instance by having threading-aware C extensions and funneling additional requests through it, so blocking calls don't just put that instance to sleep.

[different person...]

That said, I don't see any realistic rationale for threaded concurrency yet.
For instance, we need N+ processes for N cores anyway

I've been running Rails in config.threadsafe! for over a year, using MRI. Specifically to deal with longish request times, that are mostly I/O bound (waiting on third-party http API's). Also it can be a way cheaper way to scale on any platform (including self-hosted) where your cost is per-process or per-RAM unit. It is simply not true that there are no use cases for MT concurrent thread dispatching, even with the GIL. (I/O boundness means the GIL is not a significant barrier). Even with the truth that with the GIL, you still want N processes for N cores, you might want those processes to do MT concurrent request handling. (Plus people ARE deploying on rubies without the GIL).

Without any special funnelling existing requests through any special extensions, it pretty much just works. The only bugs I've run into have been AR ConnectionPool, where I've tried to help address them, with limited success. (I think what's in master/4.0 now mostly does address them, although AR/ConnectionPool's overall design is not great for multi-threaded concurrency, Sequel and DataMapper both do a lot better here in their design. I could say more).

JRuby has supported concurrent requests on the same Rails application for years...since 2.2 introduced "threadsafe" mode. Since that time, many (most?) people have opted to continue running Rails with multiple instances even on JRuby (by having multiple JRuby instances in the same process). That is not a Rails issue; that is a deployment issue...a server issue.

Well, it's both. I think one of the reasons people even in jruby (where it's pretty easy to enable) have chosen to not run with multi-threaded concurrent request handling is because Rails is perceived to be buggy under that scenario. And that perception has in part been justified.

Few MRI-possible deployment solutions have supported MT concurrent request dispatching. Possibly in part because of the same perception, and an added perception that because of the GIL MT concurrent request dispatching is not useful (not true, a misperception). However, this is starting to change -- upcoming Passenger Enterprise will support this even under MRI. thin 2.0 says it will support this even under MRI. and of course puma supports this, even under MRI.

To actually get MT concurrent request handling, you need a deploy/app server that supports it, AND you need Rails to support it. You need both. But the deploy solutions are starting to, and Rails have it on by default will send a message that Rails is serious about supporting it for real, encouraging deploy servers to support it, and resulting in more people using it, which will result in more remaining bugs and mis-designs being fixed.

If a rails app has threadsafe on, but the deploy solution doens't support it -- you won't run into any bugs related to concurrent request handling, no problem. It won't help you much, but it won't really hurt you much either (with the exception of having to pre-load more, making app startup slower. How slow ruby, esp MRI, is at require/loading is to some extent a root problem.).

It's a good idea.

(It should be noted, that indeed if you turn config.threadsafe on in development, it can wreak havok with dev-mode auto-reloading. (cache_classes false). I have run into this, yes. It can be a problem. I am not sure if there is a good solution. I end up having to give up on dev-mode class-reloading when I want threadsafe even in dev. Another option is certainly threadsafe in production but not dev).

@tessi tessi referenced this pull request Dec 20, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.