Skip to content

CookieStore session id being shared across different users #476

Closed
ericallam opened this Issue May 10, 2011 · 40 comments
@ericallam

On rails 3.1.beta1 and rack 0.3.0.beta, using the cookie session store in production. Here is what is happening:

User 1 logs into site, cookie is set and session id is set.
User 2 logs into site, same as above.
User 2 makes another request to site and the session_id is set to User 1's session_id, thus User 2 becomes logged in as User 1.

When User 1 makes subsequent requests, they still get to be logged in as User 1. But User 2's subsequent requests has User 1's session_id in the session.

I've only seen this happen with the cookie store, and only in production (could not replicate locally, even with cache_class = true).

Doesn't seem to be a client issue (happens in Chrome, Safari, and Firefox).

active_record_store does not suffer this problem.

Using this devise:

gem 'devise', :git => 'git://github.com/plataformatec/devise.git', ref: '50a5ad54ce0ccff'

I haven't been able to pinpoint where the problem is happening, but I'm guessing its probably in the Rack::Session::Cookie implementation, although I can't be sure and I figured this would still be the best place to log the issue.

Its a little tough to debug since it's only happening on production but I'm going to keep trying.

@josevalim
Ruby on Rails member

@rubymaverick if you turn cache classes off in production, does the problem still exists? It should at least make it easier to debug. Also, which is the ID of those users? Does any of them has the id in the database equal to 4? Finally, does it also fail using Rails master? I have fixed a cookie related issue after the beta release. Thanks!

@josevalim josevalim was assigned May 10, 2011
@ericallam

@josevalid: it fails using rails and rack master.

Neither of the users have an ID of 4. I logged the session out on each request and it looked something like this:

User 1:
session: {:session_id => '9e26a62dfcf374d96a7a899da4706340'}

User 2:
session: {:session_id => '9e26a62dfcf374d96a7a899da4706340'}

@josevalim
Ruby on Rails member

@rubymaverick so I am out of suspicions from where this bug is coming from. Could you please reproduce it in a smaller application and push it to Github? I will take a look at it straight away.

@josevalim
Ruby on Rails member

@rubymaverick hey mate, any news? we want to release a RC soon, so it would be cool if we have this fixed before! :D

@ericallam

@josevalim I'm having problems recreating the problem in a fresh app. I'm going to keep working on it today so hopefully by tonight I'll have something to show.

@josevalim
Ruby on Rails member

Sweet, thanks a lot!

@ericallam

@josevalid I haven't yet been able to replicate. Beginning to think its super specific to my app, and my even be attributed to omniauth/devise.

@dhh
Ruby on Rails member
dhh commented May 13, 2011

Thanks for the update. Closing this for now.

@dhh dhh closed this May 13, 2011
@stephenjudkins

We saw this issue as well. It occurred in production mode only. We are using Jruby 1.60, jruby-rack (1.0.4) w/ Jetty (7.2.2.v20101205). We're slammed this week, but we could replicate it 100% of the time. We will try to track it down further.

@josevalim josevalim reopened this May 16, 2011
@josevalim
Ruby on Rails member

Ok, if you could really track it down it would be very very important. If this turns out to be a Rails issue, this is a big blocker and we really appreciate your efforts on reproducing it!

@josevalim josevalim closed this May 16, 2011
@josevalim
Ruby on Rails member

Doh, I have closed by mistake. Reopening. :)

@josevalim josevalim reopened this May 16, 2011
@ericallam

Well what more perfect time to try and fix this than at railsconf :)

@josevalim
Ruby on Rails member

Certainly, I am totally available. Just find me and we can fix this together!

@ericallam

Think we found the issue (me and @nbibler, at it again).

When an asset is being served fresh by Rack-Cache the response will include a Set-Cookie header that sets a cookie string belonging to the user who requested that asset when it was stored in the cache. So here is the scenario from the ticket above:

User 1 makes a request to asset rails.js. There is a miss on that asset in the cache so rack-cache persists the response in the cache store with User 1's cookie string in a Set-Cookie header.

User 2 makes a request to asset rails.js. Rack-Cache checks the cache store and finds rails.js and returns a response with the Set-Cookie header from User 1.

User 2's session cookie is now User 1's.

I think this issue was probably caused by us using Sprockets to serve assets, since the request/response call travels all the way down the middleware stack when making a request on a sprocket asset. So when the original asset response is stored, it has gone through the Session middleware, which sets the Set-Cookie header that gets persisted in the cache by Rack-Cache.

This fixes the problem, although I'm not sure this is the preferred way you guys will want to fix this:

envylabs/rack-cache@56a8893

@rtomayko

Boo. I can think of a few solutions:

  1. Add an option to rack-cache that would be an array of header names to discard before either writing to the store or serving responses. It could probably even default to ['Set-Cookie'] but not without a major revision.

  2. Put a piece of middleware directly in front or directly behind rack-cache that deletes user-specific headers like Set-Cookie.

  3. Add code to the Rails and/or sprockets asset processing pipeline that disables sessions.

I think there's actually two issues here.

You probably don't want to Set-Cookie on asset responses to keep response sizes down. I'd say 3) makes a lot of sense for the asset case specifically, regardless of caching.

Separately, with Rails using rack-cache so heavily for page cache cases, I think it's important that there be some safeguards around not sending back Set-Cookie session headers to multiple clients accidentally. We'll need 1) or 2) to address that. I'm up for adding an option to rack-cache. It's a fairly frequent request and having to write custom middleware to filter headers each time is annoying. I can't think of a single case where I'd want to cache Set-Cookie response headers.

@josevalim
Ruby on Rails member

Thanks guys for finding this out. The current behavior is funny because Rails sessions should be lazily read and written, so for the assets case, you should no Set-Cookie at all. Could you please ping me tomorrow during RailsConf? I would like to check this out and see what is making the session to be set so we can solve 3).

Also, just by curiosity, why are you using the assets pipeline on production?

@josevalim
Ruby on Rails member

Fixeeeeed: 6d31224

@josevalim josevalim closed this May 17, 2011
@rmm5t
rmm5t commented Feb 2, 2012

@josevalim We're having issues somewhat related to this issue. We think it's more of a problem with Rack::Cache though. In summary, we have a few resources where public cache headers are set. These same resources are automatically picking up our Rails cookie session store too. Because Rack::Cache will cache the Set-Cookie header if the response explicitly set a public cache header, we started to experience hijacked user sessions in production after Rack::Cache served cached resources with someone else's session. I opened up an issue over in Rack::Cache to start a discussion on the topic, but also wanted to reference that issue here for others who have similar problems.

@nbibler
nbibler commented Feb 2, 2012

We just recently (about 2 weeks ago) had an issue with this after bumping an application up to Rails 3.1.3 (from 3.0.10). I had to yank public caching off of HTML pages, as well...

@nbibler
nbibler commented Feb 2, 2012

I should mention in our case that we were never able to determine which caching layer was the culprit (Rack::Cache, Varnish, or CloudFront - most unlikely). So, I wouldn't say that we're prepared to point fingers at Rack::Cache for this, although, I guess the addition of Rack::Cache was probably the only major architecture/stack change in that migration. So, it's possible.

@rmm5t
rmm5t commented Feb 2, 2012

@nbibler For us, the only caching layer we had was Rack::Cache, and the X-Rack-Cache header showed a cache hit. Furthermore, Varnish's default behavior (and by design) is to not cache any content with a Set-Cookie header. I suspect this is a common default amongst caching layers. That wiki article also shows how to instead configure Varnish to still cache all public content but also strip Set-Cookie headers beforehand.

@nbibler
nbibler commented Feb 6, 2012

Right, as I mentioned, we also have CloudFront involved... so depending on how the request was routed, it would also be caught at that level, which occurs after the Varnish level. Sadly, CloudFront isn't intelligent enough to strip out the Set-Cookie headers on cached responses, in our experience.

@rmm5t
rmm5t commented Feb 10, 2012

@nbibler, I have a pull-request over on Rack::Cache that you might find useful. Please review in hopes that it helps with the problems that you were seeing a few weeks ago.

@benpetro

Sorry to raise this issue again, but I don't know where else to go. I'm seeing behaviour that mirrors this bug in my Rails 4.2 app. I've recently upgraded from 3.2 which I was running fine on for 2 years. Since upgrading, I'm now finding issues of users becoming logged in as another user. I'm trying to debug but it happens so sporadically it's difficult to reproduce. Basically, this happens:

Person 1 is logged in. (I'm storing session[:user_id] = @user.id.to_s)
Person 2 is also logged in, on another computer in the office, however I've had other staff report that it's happened in another city, so location is not a factor.
Person 1 leaves computer for a period of time, returns to their desk, clicks a link in the app, and finds they are now logged in as Person 2.

FYI, I'm not using Devise or similar gem, but have a simple current_user method in my ApplicationController to find @current_user ||= User.find(session[:user_id])

This seems to happen to a few people more than others, and Person 1 often becomes Person 2, not Person 3 or 4 of 5 etc. This also happens across platforms (Mac, OSX, Linux - all in Chrome)

Like I said, this has ONLY started happening since upgrading to 4.2. We ran perfectly fine in Rails 3 for 2+ years.

I'm worried, and don't know where to turn.

The only other change is I've recently moved from Rackspace to hosting at Google Cloud Platform, but not sure why that would be a factor. I thought the session was completely local.

Please help, and sorry I sound so desperate.

@matthewd
Ruby on Rails member

@benpetro

I'm worried, and don't know where to turn.

The mailing list, or StackOverflow. Bring code.

@rafaelfranca
Ruby on Rails member

@benpetro the cookie storage changed between Rails 4.1 and 4.2 from marshal to json, maybe it is related? What is your cookie storage configuration?cIt should be inside config/initializers/cookies_serializer.rb

@benpetro

@rafaelfranca In my hast upgrading, it turns out I missed the cookies_serializer.rb initializer, thus my app wasn't specifying json as the serializer at all. I'm crossing my fingers that this solves my problem, thank you for your tip.

@jxdx
jxdx commented May 6, 2015

I am also having this issue in production. I'm using Rails 4.2.0. Is it correct to assume this issue is fixed in 4.2.1?

@benpetro
@jxdx
jxdx commented May 6, 2015

My cookie is store is already set to JSON as well.

@jxdx
jxdx commented May 13, 2015

Anyone else still having this issue? I'm debating now whether to downgrade rails to 4.1

@rafaelfranca
Ruby on Rails member

Could you describe how to reproduce the issue? I fear that we don't have how to help without that.

@benpetro
@jxdx
jxdx commented May 13, 2015

Yes, I have the exact same problem.

@benpetro
@jxdx
jxdx commented May 13, 2015

well this particular app I started with 4.2 but I do have other apps using 4.0 and 4.1 where I have not experienced this problem.

@jxdx
jxdx commented May 14, 2015

@benpetro Please let me know if you downgrade and it fixes the issue. Thanks!!

@jxdx
jxdx commented Jun 17, 2015

@benpetro I'm curious what your current_user method looks like?

@arronmabrey

I also migrated a rails 3.2 app to rails 4.2 (with devise 3.4.1) in the last month or two.

I'm getting the same user session switching reports from people but I have been unable to reproduce the issue myself.

It looks like none of the asset requests contain a Set-Cookie header, which make me think it's not exactly the same issue that @ericallam described.

@aldefouw
aldefouw commented Feb 8, 2016

Any chance that this problem is only showing up in production environments because of the application server you're using?

Phusion Passenger's turbocaching feature (which is enabled by default) can get you into trouble if you are not careful. Passenger 5 includes an HTTP caching layer.

If you are using Passenger, take care that you ensure all of your headers are being sent as "cache-control: private." By default, Rails sends them as 'private' - but if you modify them, like we did, that's where you can get into trouble.

I asked about this issue on Stack Overflow, and someone pointed out to me that turbocaching was the likely culprit.

http://stackoverflow.com/questions/35234161/unintentional-session-hijacking-in-rails-4-2-4-devise-warden-phusion-passen

Hope this helps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.