CSRF protection prevents some webkit users from submitting forms

[zetter]

Hi,

We've recently been investigating reports from our users that they are unable to submit forms.

Upon investigation it appears that browsers can get in a state where Rail's CSRF (Cross-Site Request Forgery) protection stops the form being submitted.

To reproduce

It's possible to produce a minimal Rails app which has this problem:

rails new csrf-test
cd csrf-test
bundle exec rails generate scaffold Test test:string
bundle exec rake db:migrate
bundle exec rails server

How to replicate it on mobile Safari (tested on iOS9):

• Load a page containing a form (will be http://localhost:3000/tests/new in this example).
• Quit Safari by double-tap the home button and swipe up.
• Open Safari from the home screen. You should see the same page with the form.
• Submit the form.

You will see the Rails invalid authenticity token error- this is a "The change you wanted was rejected" message in production, or an ActionController::InvalidAuthenticityToken in development. I've also made a video that follows theese steps.

How to replicate on Desktop Safari (tested on Safari 9.0 on OSX)

• Go to 'Safari' > 'Preferences...' > 'General' and set 'Safari opens with:' to 'All windows from last session'.
• Load a page containing a form (will be http://localhost:3000/tests/new in this example).
• Quit Safari (with CMD+Q)
• Open Safari. You should see the same page with the form.
• Submit the form.

This problem seems to happen regardless if:

• the app is served HTTP or HTTPS
• the app's environment is development or production
• the browser is manually quit by the user, or quit by the OS (to save memory)

I have also been able to replicate on Chrome on Android. I haven't yet been able to replicate it on Chrome and Firefox on OSX using their 'restore tabs' options like I did in Safari. There may be other browsers that are affected.

What's happening

Looking at the Rails logs, and the cookie submitted by the browser I believe that the browsers are caching the page, but clearing session cookies. This means the form has a authenticity_token parameter, but the Rails session cookie has been cleared so has no corresponding _csrf_token.

Here is a annotated log showing this:

# Browser loads the form for the first time
Started GET "/tests/new" for 127.0.0.1 at 2015-10-13 09:23:18 +0100
  ActiveRecord::SchemaMigration Load (0.1ms)  SELECT "schema_migrations".* FROM "schema_migrations"
Processing by TestsController#new as HTML
  Rendered tests/_form.html.erb (37.0ms)
  Rendered tests/new.html.erb within layouts/application (41.4ms)
Completed 200 OK in 256ms (Views: 243.3ms | ActiveRecord: 0.3ms)

# (Asset requests ommited)

# Browser quits, clearing session cookies
# Browser re-opens, reloads the page from cache without doing a request

# Browser posts the form:
Started POST "/tests" for 127.0.0.1 at 2015-10-13 09:23:37 +0100
Processing by TestsController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"IhsNUyL6Y/riLIujH+ExkTZN9pEPfwAVVB/t9pwrnkIR6lw1bAl3ZFY+bPg+zqMf3pj3qeY0vgbKblrWgr0vnQ==", "test"=>{"test"=>""}, "commit"=>"Create Test"}
Can't verify CSRF token authenticity
Completed 422 Unprocessable Entity in 1ms (ActiveRecord: 0.0ms)

ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
  actionpack (4.2.4) lib/action_controller/metal/request_forgery_protection.rb:181:in `handle_unverified_request'
  actionpack (4.2.4) lib/action_controller/metal/request_forgery_protection.rb:209:in `handle_unverified_request'
  actionpack (4.2.4) lib/action_controller/metal/request_forgery_protection.rb:204:in `verify_authenticity_token'
  # (Stack trace truncated)

I've tried to find more documentation about this kind of caching that browsers do. I found this article about the WebKit Page Cache but it appears to be out of date (it says HTTPS pages do not use the Page Cache, but I have seen this problem on HTTP and HTTPS pages). If anyone can find more about this I'd love to know.

I'd like to know:

• Have others seen this problem? I haven't been able to find reported before, or find any documentation regarding this, but perhaps I have missed something.
• Is there a way that Rails could be changed that would prevent this happening?
• Are there any workarounds for this that we could apply in our application?

Thanks for any help.

[rafaelfranca]

Is there a way that Rails could be changed that would prevent this happening?

I can't see any way to prevent this happening without opening the application to possible CSRF attacks.

[pixeltrix]

@zetter does changing the Cache-Control header to no-store, no-cache make any difference?

[zetter]

@zetter does changing the Cache-Control header to no-store, no-cache make any difference?

I've added this line in the application config:

config.action_dispatch.default_headers.merge!('Cache-Control' => 'no-store, no-cache')

Mobile Safari has the same problem, however I can no longer reproduce this in Desktop Safari and Chrome on Android.

it does feel odd to me that the page is no-store but this isn't respected by Mobile Safari but is by the other browsers. Perhaps this is a bug in Mobile Safari although, as I said above I found it very difficult to find documentation around the expected behaviour of this kind of page caching. Perhaps there is another combination of cache headers that makes this or another work around for Mobile Safari?

UPDATE: I found that Mobile Safari does respect the 'no-store' header, however adding the header and refreshing the page isn't enough. For it to take effect also had to clear the cache using the 'Clear History and Website Data' option from Safari Settings.

This problem might also be avoided if the CSRF token was in a persistent cookie, but I'm not sure if there are security implications for doing so. It would be really helpful if anyone could explain to me if this would be fine, or a dangerous thing, to do.

[alepore]

interesting topic, i see this error quite often.
@zetter here there's some mention about CSRF token in cookies, but it's used with javascript for caching purposes https://www.fastly.com/blog/Caching-the-Uncacheable-CSRF-security

[zetter]

An update:

• I've deployed the change to serve all pages of our application to use a 'no-store' header. We're tracking how many times we see invalid authenticity token errors so next week I should be able to report how effective this change was for us.
• I've tried working round the issue in Mobile Safari by experimenting with headers and client side JS (such as hooking into load/unload events) but have been unable to. I plan to raise a bug with Webkit/Apple, and will link to this when I do.
• I'm curious how other frameworks handle this problem, for example, I know that Django uses a similar mechanism for CSRF prevention so will check to see if they have already solved this problem.

@alepore Thanks for sharing. I'm not sure if any of the solutions there help since it's they are about solving the problem of being able to use CSRF prevention token in combination with server-side caching.

[zetter]

My investigation into Django:

Django uses a similar mechanism to rails to prevent CSRF attacks- a token is stored in a cookie is compared to a tokens submitted with a form. What is different is how they store the token in a cookie in the default case:

Comparing Rails and Django

Rails adds the token to the session cookie under the _csrf_token key. Since the Rails session cookie has no expiry set, it will be cleared when the browser closes.

Django puts adds the token in it's own cookie called CSRF_COOKIE. This is a persistent cookie that expires in a year. If subsequent requests are made, the cookie's expiry is updated.

Why does Rails behave in this way?

Rail's CSRF protection comes from the csrf_killer plugin. The plugin had the following warning:

Make sure the session cookies that Rails creates are non-persistent. Check in Firefox and look for "Expires: at end of session"

This warning got imported into Rails from the csrf_killer documentation in 2c73115. It has since been replaced in 7d8474e by this warning

It is common to use persistent cookies to store user information, with cookies.permanent for example. In this case, the cookies will not be cleared and the out of the box CSRF protection will not be effective.

Read full section of the guide. I'm not sure if "the cookies will not be cleared" in the statement above is referring to when a user signs out, when a session ends, or when an invalid authenticity token is given.

Why does Django behave in this way?

In contrast, it looks like Django made the explicit decision to store CSRF prevention token in a permanent cookie. From the Django docs:

The reason for setting a long-lived expiration time is to avoid problems in the case of a user closing a browser or bookmarking a page and then loading that page from a browser cache. Without persistent cookies, the form submission would fail in this case.

You can also read the bug which caused this change in Django, a question on the Django mailing list asking if persistent cookies are safe and a related bug for how Internet Explorer can be configured to block persistent cookies.

Is it secure to store the CSRF prevention token in a separate permanent cookie?

Storing the token in a permanent cookie would fix the original issue as it would no longer expire when the browser closed. It looks like Django has made the choice that it is. I couldn't find any information around CSRF attacks that suggests it wouldn't be. The only item I found was on the OWASP Wiki which suggests that the shorter the lifespan of the token the better, without any justification:

To further enhance the security of this proposed design, consider randomizing the CSRF token parameter name and or value for each request. Implementing this approach results in the generation of per-request tokens as opposed to per-session tokens. Note, however, that this may result in usability concerns. For example, the "Back" button browser capability is often hindered as the previous page may contain a token that is no longer valid. Interaction with this previous page will result in a CSRF false positive security event at the server.

I'd love to know from anyone who knows more about CSRF attacks to find out if it would be safe to store the CSRF prevention token in a permanent cookie (both for our application, and for Rails).

[jeremy]

Thank you for the investigation and writeup, @zetter. Agree with Django's reasoning and decision 👍 May be a bit tricky to introduce to Rails apps in a compatible way, though.

[aganov]

I can confirm the issue, combination El Capitan + Safari + http2, not a single working form... However there is no problem with Safari on Yosemite...

Edit: after disabling HTTP/2, the problem disappears

[zetter]

Another update:

Here's a graph of showing the number of our users seeing form authenticity token errors as a proportion of the number of visits to our site. It looks like the rate of errors have dropped since we introduced no-store cache header.

Note that the traffic pattern to our site is unusual as a lot of our users visit weekly, this means they may have a page open in a browser from a visit a previous week. I think that's why even though there was an initial drop in errors, another drop came a week later as the cohort of the previous weeks refreshed the pages and got the updated headers.

I still plan to:

• raise a bug with Safari/Webkit about the inconsistent behaviour of no-store as compared to other browsers (UPDATE: I found Safari is respecting the no-store header, it just needs to be reset- see above comment for more)
• try storing the token as a permanent cookie in our app to see if error rates are further decreased

[aganov]

@zetter Just wondering, is your server HTTP/2 enabled?

[zetter]

@zetter Just wondering, is your server HTTP/2 enabled?

@aganov It is not, and I have not tested my original example with a HTTP/2 server.

My understanding of HTTP/2 is that there are no changes to cookies or headers so the behaviour I am seeing shouldn't change. You said before that there wasn't a single working form- were you following my original reproduction example? or is this another app that might have a different problem?

[javan]

try storing the token as a permanent cookie in our app to see if error rates are further decreased

You can try that by giving your session cookie an expire_after value to persist it across browsing sessions:

Rails.application.config.session_store :cookie_store, key: '_myapp_session', expire_after: 2.weeks

You'll need to consider possible security implications based on your session data. In my case, it solved the problem.

+1 for moving the CSRF token to its own permanent cookie.

[btoews]

I saw a case a few years ago where the Django CSRF behavior resulted in a security vuln for an app. With an XSS on foo.example.com, an attacker was able to set the CSRF token cookie for .example.com causing it to be sent with requests to example.com and *.example.com. The attacker-controlled CSRF token clobbered the one that had been set by the site at example.com. Because the attacker knows the new CSRF token value, they can then perform CSRF attacks.

This sort of attack doesn't affect Rails applications because the CSRF token is stored in a signed cookie. Most applications will reset_session during login, so even if an attacker can set an unauthenticated cookiestore cookie (session fixation), the user will receive a new CSRF token after authenticating.

Django's CSRF token storage seems like a pretty big weakness to me. If you decide to store Rails CSRF tokens in a separate cookie, they should definitely still be signed.

As for the security impact of using a permanent cookie, the risk seems the same as using a permanent cookie for the whole cookiestore session. The CSRF token is generally the most sensitive thing stored in a session, so if there's anything that should be deleted when the browser is closed it's that. This issue seems like a Safari bug to me. Safari is notoriously bad at handling cookies. I think suggesting that apps work around the issue by using a permanent cookie for the session is better than working around the Safari bug in the framework.

[ctpelnar1988]

Try changing "protect_from_forgery with: :exception" to=> "protect_from_forgery with: :null_session"

:null_session - Provides an empty session during request but doesn't reset it completely. Used as default if :with option is not specified.

[bcardiff]

For me, using :null_session, together with some straight devise usage, it ended up in an infinite redirection loop in an Safari/iPad. For the time being I am using Chrome for iPad.

[mikebaldry]

Any thoughts or update on this? I think its hidden a lot in the default mode, as people just get a null session and we quietly log it. I've switched to raise and now I can see this is affecting a fair few people.

[chrisnicola]

I've been seeing something similar since switching to the :exception behaviour. Specifically from unauthenticated endpoints. We are also mostly an Angular/JSON app so CSRF is a bit lower value for us but we like to keep in enabled.

I've switched to the :reset_session behaviour on any unauthenticated endpoints such as creating a new user or logging in which seems sensible and doesn't cause any problems.

[pixeltrix]

Urgh, looks like we're seeing this on https://petition.parliament.uk. However it looks like we're seeing it across multiple browsers and not limited to Mobile Safari.

[Bramjetten]

We were seeing a lot of InvalidAuthenticityToken exceptions. After reading this thread and further inspection I saw that the exceptions were only caused by mobile browsers. Adding 'Cache-Control' => 'no-store, no-cache' seems to have fixed it. Immediate drop in exceptions. Is this something we always have to account for or is there a better solution?

[jeremy]

@Bramjetten That means no browser caching, though. Using a persistent cookie for your session is prob a more reasonable fix for you.

[Bramjetten]

@jeremy That's what I did at first, but it didn't seem to help.

[jeremy]

May need to give it some time as people establish new sessions?
On Mon, Apr 4, 2016 at 09:27 Bram Jetten notifications@github.com wrote:

@jeremy https://github.com/jeremy That's what I did at first, but it
didn't seem to help.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#21948 (comment)

[lucascaton]

Hey @rafaelfranca, could you please shed some light?
Sorry if you aren't the right person to @ in this case.

Thank you.

[BenFenner]

If you need extremely easy steps to reproduce on any browser and any Rails site using protect_from_forgery with: exception, including GitHub itself, check my previous post:
#21948 (comment)

All of our sites are on Rails 5.2.2 now, so I can check there, but it's going to be the same story.

[pixeltrix]

@BenFenner out of curiosity what would you expect it to do in your example? AFAICS there are two options:

1. Ignore the CSRF token error or make the token valid somehow and process the request
2. Redirect to the login page

Option 1 is a massive security hole so that just leaves option 2 which is what you'll get when you change protect_from_forgery to use :null_session.

[rdlugosz]

The problem with approach #2 is that these users may not expect to have been logged out of the site and will get annoyed that they're frequently being asked to log in again. A third option exists: redirect to home with a message asking that they retry their request. This gets the token refreshed, prevents a potentially insecure action, and allows the user to continue without having to re-login.

[pixeltrix]

@rdlugosz that's an app-level concern, not something we can do at the framework level, e.g.

class PostsController < ApplicationController
  rescue_from ActionController::InvalidAuthenticityToken do
    redirect_to root_url, alert: "Your request has expired, please try again"
  end
end

I think the best we can do to fix this is to add a section to the security guide on the causes and how you can deal with the problem.

[travisp]

It seems that there may be some approaches to deal with this that might feel better to the user, for example:

1. use an approach that logs out the user in all tabs when logged out in one (achievable with websockets, localstorage events)
2. update CSRF tokens or refresh the session in all tabs when session changes (again with websockets or localstorage events).

But these solutions feel probably beyond the scope of this issue as the exact desired behavior might be application specific. It would be helpful to at least describe this issue in the security guide so that developers stumbling upon this can understand what's happening and make an informed choice.

Does this issue exist only when users are logging in/out and they have multiple tabs open, or are there other scenarios it could happen?

[rdlugosz]

I agree that updating the docs may be the best resolution to this, failing the proper vetting and implementation of a persistent cookie like Django uses as described earlier (which I'm not qualified to do; that needs careful attention from the sec team).

One issue in this thread is that two different scenarios have gotten co-mingled. The original issue has to do with mobile browsers not persisting the CSRF state between extended sessions (i.e., I have a tab open in safari on my iphone, leave it for a while, and then when I come back the next search I submit to the app causes an invalid token error). Later, a separate scenario was described that involves multiple tabs and logging out in one of them... that's an entirely different situation than what started this issue.

So @travisp, no - it isn't really about multiple tabs, although there may be a related case that involves them.

[monroemann]

We can confirm that at least one of our users is having this same issue right now. We haven't yet been able to solve it.

[Richard-Degenne]

Hi, I've been following this issue for quite some time now, and I'm just wondering.

To developers suffering from this issue, does your project use Devise for authentication?

Please react to this comment with 👍 for yes, 👎 for no.

[AlexanderZaytsev]

A heads-up to someone trying to find out why current_user is being nil inside your Devise::RegistrationsController even though it calls autheticate_user! behind the scenes for actions like update.

Basically, I create guest accounts for users to try out my service (I have an attribute role=guest in my User model). After some time, I display a form suggesting users to "save" their account by entering an email and a password. For years I couldn't understand how this code would result in NoMethodError: undefined method id' for nil:NilClass`:

class Users::RegistrationsController < Devise::RegistrationsController
  def update
    @user = User.find(current_user.id)
    # Update @user from role=guest to role=user, etc
  end
end

So it turns out, here's what caused the error:

1. I create a guest account for a user.
2. Some time later I display a form and ask the user to enter their email & password.
3. At this step, the user closes/switches from their iOS browser to another app.
4. Some time later, the user opens their iOS browser again, sees the old form and tries entering email & password and submitting it.
5. It results in an invalid CSRF token, which results in current_user being nil inside the update action. Also, this resulted in the user's session being reset, resulting in a loss of the their guest account & data they entered.

The fix I'm using now:
Raise an exception in ApplicationController

class ApplicationController < ActionController::Base
  protect_from_forgery prepend: true, with: :exception
end

And rescue it inside Users::RegistrationsController

class Users::RegistrationsController < Devise::RegistrationsController
  rescue_from ActionController::InvalidAuthenticityToken do
    redirect_to request.referrer, alert: "Your request has expired, please try again"
  end
end

This way, the user stays logged in, sees "Your request has expired, please try again" and is presented with a fresh form with a valid csrf token.

It took me a few years to hunt down this bug until I got fed up with it today :) Hopefully this info will help someone with a similar problem.

[pixeltrix]

So it turns out that Firefox may autocomplete hidden fields:

https://bugzilla.mozilla.org/show_bug.cgi?id=520561

Has anyone seen this as a cause of 422 errors? For example someone could log out in one tab and then hit refresh in another which may result in an stale authenticity_token being injected into the hidden field.

[mindaslab]

It's not a Rails problem if you are using nginx, puma along with certbot. Its happening because certain stuff to authenticate is not available for the Rails app, before reaching Rails it gets filtered out.

To fix it you can see this https://stackoverflow.com/questions/39012356/devise-cant-verify-csrf-token-authenticity-the-https-enabled-on-server-no-jso

[MattyMc]

I've recently had a huge spike in ActionController::InvalidAuthenticityToken errors due to what I believe is browser HTML caching resulting in requests without the session cookie (as described in this thread).

I'm testing a solution that:

1. Sets the XSRF-TOKEN as an unsigned cookie as described in this SO answer by u/HungYuHei. Relevant code:

# application_controller.rb
class ApplicationController < ActionController::Base
  after_action :set_csrf_cookie_for_ng
  
  private

  # 
  def set_csrf_cookie_for_ng
    cookies['XSRF-TOKEN'] = form_authenticity_token if protect_against_forgery?
  end

  # Optional. Described more below.
  # Extend valid_authenticity_token? to also check XSRF-TOKEN
  def verified_request?
    super || valid_authenticity_token?(session, request.headers['X-XSRF-TOKEN'])
  end
end

2. Uses some javascript to check whether the above cookie is set. The above cookie along with the session cookie will be dropped if the user closes the browser. Since the session cookie is HTTP only we can't read it with javascript. However we can read XSRF-TOKEN that we set and if it's not there we can force a browser refresh:

// app/javascript/packs/application.js
const hasCrossSiteReferenceToken = () => document.cookie.indexOf('XSRF-TOKEN') > -1;

if (!hasCrossSiteReferenceToken()) {
    location.reload();
}

A few notes:

1. I'm definitely concerned about DOS'ing my own server here. But so far, it hasn't happened. I can't see a way that it would.
2. You don't have to set a XSRF-TOKEN in the cookie... it could be anything (timestamp, for example). Just make sure it's set with every non-protected request to your site.

Would welcome comments.

[vizcay]

I just launched a new Rails v6.0.2.2 application a month ago with a 85% mobile user base and I get a consistent bunch of InvalidAuthenticityTokens errors every day (I will say less than 1% of all requests anyway) but bothers me for the UX of these people.

Just finished reading all thread looking for mitigations. This issue is still relevant 5 years later!

[augnustin]

Interesting topic.

I agree that the issue is still relevant.

I now understand better in which situation does the issue arises. I'm no security guru, and still am not very clear on what situation does the csrf token prevents attacks but I understand this has to do with current page freshness, hasn't it?

Couldn't we imagine something like another hidden field, along the x-csrf-token one, that would be token-generated-at: timestamp, and a rails_ujs a script that does like:

$(window).on('focus', e => {
  if $('[name=token-generated-at]').val() > specific duration {
    window.location.reload(false);
  }
});

Or if this has to do with the presence of a session cookie, this should be also checkable with JS, shouldn't it?

This would be a lesser optimized UX for those users but at least they wouldn't have any error!