CSRF protection prevents some webkit users from submitting forms

[zetter]

Hi,

We've recently been investigating reports from our users that they are unable to submit forms.

Upon investigation it appears that browsers can get in a state where Rail's CSRF (Cross-Site Request Forgery) protection stops the form being submitted.

To reproduce

It's possible to produce a minimal Rails app which has this problem:

rails new csrf-test
cd csrf-test
bundle exec rails generate scaffold Test test:string
bundle exec rake db:migrate
bundle exec rails server

How to replicate it on mobile Safari (tested on iOS9):

• Load a page containing a form (will be http://localhost:3000/tests/new in this example).
• Quit Safari by double-tap the home button and swipe up.
• Open Safari from the home screen. You should see the same page with the form.
• Submit the form.

You will see the Rails invalid authenticity token error- this is a "The change you wanted was rejected" message in production, or an ActionController::InvalidAuthenticityToken in development. I've also made a video that follows theese steps.

How to replicate on Desktop Safari (tested on Safari 9.0 on OSX)

• Go to 'Safari' > 'Preferences...' > 'General' and set 'Safari opens with:' to 'All windows from last session'.
• Load a page containing a form (will be http://localhost:3000/tests/new in this example).
• Quit Safari (with CMD+Q)
• Open Safari. You should see the same page with the form.
• Submit the form.

This problem seems to happen regardless if:

• the app is served HTTP or HTTPS
• the app's environment is development or production
• the browser is manually quit by the user, or quit by the OS (to save memory)

I have also been able to replicate on Chrome on Android. I haven't yet been able to replicate it on Chrome and Firefox on OSX using their 'restore tabs' options like I did in Safari. There may be other browsers that are affected.

What's happening

Looking at the Rails logs, and the cookie submitted by the browser I believe that the browsers are caching the page, but clearing session cookies. This means the form has a authenticity_token parameter, but the Rails session cookie has been cleared so has no corresponding _csrf_token.

Here is a annotated log showing this:

# Browser loads the form for the first time
Started GET "/tests/new" for 127.0.0.1 at 2015-10-13 09:23:18 +0100
  ActiveRecord::SchemaMigration Load (0.1ms)  SELECT "schema_migrations".* FROM "schema_migrations"
Processing by TestsController#new as HTML
  Rendered tests/_form.html.erb (37.0ms)
  Rendered tests/new.html.erb within layouts/application (41.4ms)
Completed 200 OK in 256ms (Views: 243.3ms | ActiveRecord: 0.3ms)

# (Asset requests ommited)

# Browser quits, clearing session cookies
# Browser re-opens, reloads the page from cache without doing a request

# Browser posts the form:
Started POST "/tests" for 127.0.0.1 at 2015-10-13 09:23:37 +0100
Processing by TestsController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"IhsNUyL6Y/riLIujH+ExkTZN9pEPfwAVVB/t9pwrnkIR6lw1bAl3ZFY+bPg+zqMf3pj3qeY0vgbKblrWgr0vnQ==", "test"=>{"test"=>""}, "commit"=>"Create Test"}
Can't verify CSRF token authenticity
Completed 422 Unprocessable Entity in 1ms (ActiveRecord: 0.0ms)

ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
  actionpack (4.2.4) lib/action_controller/metal/request_forgery_protection.rb:181:in `handle_unverified_request'
  actionpack (4.2.4) lib/action_controller/metal/request_forgery_protection.rb:209:in `handle_unverified_request'
  actionpack (4.2.4) lib/action_controller/metal/request_forgery_protection.rb:204:in `verify_authenticity_token'
  # (Stack trace truncated)

I've tried to find more documentation about this kind of caching that browsers do. I found this article about the WebKit Page Cache but it appears to be out of date (it says HTTPS pages do not use the Page Cache, but I have seen this problem on HTTP and HTTPS pages). If anyone can find more about this I'd love to know.

I'd like to know:

• Have others seen this problem? I haven't been able to find reported before, or find any documentation regarding this, but perhaps I have missed something.
• Is there a way that Rails could be changed that would prevent this happening?
• Are there any workarounds for this that we could apply in our application?

Thanks for any help.

[rafaelfranca]

Is there a way that Rails could be changed that would prevent this happening?

I can't see any way to prevent this happening without opening the application to possible CSRF attacks.

[pixeltrix]

@zetter does changing the Cache-Control header to no-store, no-cache make any difference?

[zetter]

@zetter does changing the Cache-Control header to no-store, no-cache make any difference?

I've added this line in the application config:

config.action_dispatch.default_headers.merge!('Cache-Control' => 'no-store, no-cache')

Mobile Safari has the same problem, however I can no longer reproduce this in Desktop Safari and Chrome on Android.

it does feel odd to me that the page is no-store but this isn't respected by Mobile Safari but is by the other browsers. Perhaps this is a bug in Mobile Safari although, as I said above I found it very difficult to find documentation around the expected behaviour of this kind of page caching. Perhaps there is another combination of cache headers that makes this or another work around for Mobile Safari?

UPDATE: I found that Mobile Safari does respect the 'no-store' header, however adding the header and refreshing the page isn't enough. For it to take effect also had to clear the cache using the 'Clear History and Website Data' option from Safari Settings.

This problem might also be avoided if the CSRF token was in a persistent cookie, but I'm not sure if there are security implications for doing so. It would be really helpful if anyone could explain to me if this would be fine, or a dangerous thing, to do.

[alepore]

interesting topic, i see this error quite often.
@zetter here there's some mention about CSRF token in cookies, but it's used with javascript for caching purposes https://www.fastly.com/blog/Caching-the-Uncacheable-CSRF-security

[zetter]

An update:

• I've deployed the change to serve all pages of our application to use a 'no-store' header. We're tracking how many times we see invalid authenticity token errors so next week I should be able to report how effective this change was for us.
• I've tried working round the issue in Mobile Safari by experimenting with headers and client side JS (such as hooking into load/unload events) but have been unable to. I plan to raise a bug with Webkit/Apple, and will link to this when I do.
• I'm curious how other frameworks handle this problem, for example, I know that Django uses a similar mechanism for CSRF prevention so will check to see if they have already solved this problem.

@alepore Thanks for sharing. I'm not sure if any of the solutions there help since it's they are about solving the problem of being able to use CSRF prevention token in combination with server-side caching.

[zetter]

My investigation into Django:

Django uses a similar mechanism to rails to prevent CSRF attacks- a token is stored in a cookie is compared to a tokens submitted with a form. What is different is how they store the token in a cookie in the default case:

Comparing Rails and Django

Rails adds the token to the session cookie under the _csrf_token key. Since the Rails session cookie has no expiry set, it will be cleared when the browser closes.

Django puts adds the token in it's own cookie called CSRF_COOKIE. This is a persistent cookie that expires in a year. If subsequent requests are made, the cookie's expiry is updated.

Why does Rails behave in this way?

Rail's CSRF protection comes from the csrf_killer plugin. The plugin had the following warning:

Make sure the session cookies that Rails creates are non-persistent. Check in Firefox and look for "Expires: at end of session"

This warning got imported into Rails from the csrf_killer documentation in 2c73115. It has since been replaced in 7d8474e by this warning

It is common to use persistent cookies to store user information, with cookies.permanent for example. In this case, the cookies will not be cleared and the out of the box CSRF protection will not be effective.

Read full section of the guide. I'm not sure if "the cookies will not be cleared" in the statement above is referring to when a user signs out, when a session ends, or when an invalid authenticity token is given.

Why does Django behave in this way?

In contrast, it looks like Django made the explicit decision to store CSRF prevention token in a permanent cookie. From the Django docs:

The reason for setting a long-lived expiration time is to avoid problems in the case of a user closing a browser or bookmarking a page and then loading that page from a browser cache. Without persistent cookies, the form submission would fail in this case.

You can also read the bug which caused this change in Django, a question on the Django mailing list asking if persistent cookies are safe and a related bug for how Internet Explorer can be configured to block persistent cookies.

Is it secure to store the CSRF prevention token in a separate permanent cookie?

Storing the token in a permanent cookie would fix the original issue as it would no longer expire when the browser closed. It looks like Django has made the choice that it is. I couldn't find any information around CSRF attacks that suggests it wouldn't be. The only item I found was on the OWASP Wiki which suggests that the shorter the lifespan of the token the better, without any justification:

To further enhance the security of this proposed design, consider randomizing the CSRF token parameter name and or value for each request. Implementing this approach results in the generation of per-request tokens as opposed to per-session tokens. Note, however, that this may result in usability concerns. For example, the "Back" button browser capability is often hindered as the previous page may contain a token that is no longer valid. Interaction with this previous page will result in a CSRF false positive security event at the server.

I'd love to know from anyone who knows more about CSRF attacks to find out if it would be safe to store the CSRF prevention token in a permanent cookie (both for our application, and for Rails).

[jeremy]

Thank you for the investigation and writeup, @zetter. Agree with Django's reasoning and decision 👍 May be a bit tricky to introduce to Rails apps in a compatible way, though.

[aganov]

I can confirm the issue, combination El Capitan + Safari + http2, not a single working form... However there is no problem with Safari on Yosemite...

Edit: after disabling HTTP/2, the problem disappears

[zetter]

Another update:

Here's a graph of showing the number of our users seeing form authenticity token errors as a proportion of the number of visits to our site. It looks like the rate of errors have dropped since we introduced no-store cache header.

Note that the traffic pattern to our site is unusual as a lot of our users visit weekly, this means they may have a page open in a browser from a visit a previous week. I think that's why even though there was an initial drop in errors, another drop came a week later as the cohort of the previous weeks refreshed the pages and got the updated headers.

I still plan to:

• raise a bug with Safari/Webkit about the inconsistent behaviour of no-store as compared to other browsers (UPDATE: I found Safari is respecting the no-store header, it just needs to be reset- see above comment for more)
• try storing the token as a permanent cookie in our app to see if error rates are further decreased

[aganov]

@zetter Just wondering, is your server HTTP/2 enabled?

[zetter]

@zetter Just wondering, is your server HTTP/2 enabled?

@aganov It is not, and I have not tested my original example with a HTTP/2 server.

My understanding of HTTP/2 is that there are no changes to cookies or headers so the behaviour I am seeing shouldn't change. You said before that there wasn't a single working form- were you following my original reproduction example? or is this another app that might have a different problem?

[javan]

try storing the token as a permanent cookie in our app to see if error rates are further decreased

You can try that by giving your session cookie an expire_after value to persist it across browsing sessions:

Rails.application.config.session_store :cookie_store, key: '_myapp_session', expire_after: 2.weeks

You'll need to consider possible security implications based on your session data. In my case, it solved the problem.

+1 for moving the CSRF token to its own permanent cookie.

[mastahyeti]

I saw a case a few years ago where the Django CSRF behavior resulted in a security vuln for an app. With an XSS on foo.example.com, an attacker was able to set the CSRF token cookie for .example.com causing it to be sent with requests to example.com and *.example.com. The attacker-controlled CSRF token clobbered the one that had been set by the site at example.com. Because the attacker knows the new CSRF token value, they can then perform CSRF attacks.

This sort of attack doesn't affect Rails applications because the CSRF token is stored in a signed cookie. Most applications will reset_session during login, so even if an attacker can set an unauthenticated cookiestore cookie (session fixation), the user will receive a new CSRF token after authenticating.

Django's CSRF token storage seems like a pretty big weakness to me. If you decide to store Rails CSRF tokens in a separate cookie, they should definitely still be signed.

As for the security impact of using a permanent cookie, the risk seems the same as using a permanent cookie for the whole cookiestore session. The CSRF token is generally the most sensitive thing stored in a session, so if there's anything that should be deleted when the browser is closed it's that. This issue seems like a Safari bug to me. Safari is notoriously bad at handling cookies. I think suggesting that apps work around the issue by using a permanent cookie for the session is better than working around the Safari bug in the framework.

[ctpelnar1988]

Try changing "protect_from_forgery with: :exception" to=> "protect_from_forgery with: :null_session"

:null_session - Provides an empty session during request but doesn't reset it completely. Used as default if :with option is not specified.

[bcardiff]

For me, using :null_session, together with some straight devise usage, it ended up in an infinite redirection loop in an Safari/iPad. For the time being I am using Chrome for iPad.

[mikebaldry]

Any thoughts or update on this? I think its hidden a lot in the default mode, as people just get a null session and we quietly log it. I've switched to raise and now I can see this is affecting a fair few people.

[chrisnicola]

I've been seeing something similar since switching to the :exception behaviour. Specifically from unauthenticated endpoints. We are also mostly an Angular/JSON app so CSRF is a bit lower value for us but we like to keep in enabled.

I've switched to the :reset_session behaviour on any unauthenticated endpoints such as creating a new user or logging in which seems sensible and doesn't cause any problems.

[pixeltrix]

Urgh, looks like we're seeing this on https://petition.parliament.uk. However it looks like we're seeing it across multiple browsers and not limited to Mobile Safari.

[Bramjetten]

We were seeing a lot of InvalidAuthenticityToken exceptions. After reading this thread and further inspection I saw that the exceptions were only caused by mobile browsers. Adding 'Cache-Control' => 'no-store, no-cache' seems to have fixed it. Immediate drop in exceptions. Is this something we always have to account for or is there a better solution?

[jeremy]

@Bramjetten That means no browser caching, though. Using a persistent cookie for your session is prob a more reasonable fix for you.

[Bramjetten]

@jeremy That's what I did at first, but it didn't seem to help.

[jeremy]

May need to give it some time as people establish new sessions?
On Mon, Apr 4, 2016 at 09:27 Bram Jetten notifications@github.com wrote:

@jeremy https://github.com/jeremy That's what I did at first, but it
didn't seem to help.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#21948 (comment)

[tinbka]

This MAY be related to load balancer/reverse proxy stack misconfiguration when request.ip != request.remote_ip condition occurs.

[clemens]

Based on @typeoneerror's suggestion, here's how I worked around this issue:

class SessionsController < Devise::SessionsController
  rescue_from ActionController::InvalidAuthenticityToken, with: :warn_session_reset

  private

  def warn_session_reset
    redirect_back(
      fallback_location: root_path,
      alert: 'We had to block your login attempt because your session has expired. Please try again.'
    )
  end
end

Note that I'm only doing this in the sessions controller because that's where most people seem to be experiencing the error. It's not a perfect solution but it works for me so far.

Also, frankly, I don't care that it shows a "wrong" error message to real CSRF attacks: It prevents access, so it does its job.

[elquimista]

@clemens this seems to be a good fit for me for now. I'm not sure why rails does not reset session on csrf token verification failure and raises an exception by default.

[rails-bot]

This issue has been automatically marked as stale because it has not been commented on for at least three months.
The resources of the Rails team are limited, and so we are asking for your help.
If you can still reproduce this error on the 5-1-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.

[killthekitten]

To sum it up: it didn't stop failing after we deployed the monkey patch presented in my PR (#27689) but it did prevent some class of failures.

There could be many sources of these errors, and mostly, I think, that's related to your client JS. I.e. after removing several AJAX calls from the front-end, the number of errors has significantly reduced.

We decided not to spend more time and leave it as it is.

[johnmcdowall]

Can confirm that this is still an issue on Rails 5.1.4.

Deployed an application for the first time on Friday, and have received 7 ActionController::InvalidAuthenticityToken exception reports so far. Most seem to be webkit based mobile browsers (some iOS, some Android), but there are a couple of Deskop Mozilla ones in there:

* HTTP Method: POST
* HTTP_USER_AGENT : Mozilla/5.0 (Linux; Android 8.1.0; Nexus 5X Build/OPP6.171019.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.84 Mobile Safari/537.36

---

* HTTP Method: POST
* HTTP_USER_AGENT : Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:57.0) Gecko/20100101 Firefox/57.0

---

* HTTP Method: POST
* HTTP_USER_AGENT : Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 XL Build/OPD3.170816.023) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.71 Mobile Safari/537.36

---

* HTTP Method: POST
* HTTP_USER_AGENT : Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_1 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Version/11.0 Mobile/15B150 Safari/604.1

---

* HTTP Method: POST
* HTTP_USER_AGENT : Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.98 Safari/537.36

---

* HTTP Method: PATCH
* HTTP_USER_AGENT : Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_1 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Version/11.0 Mobile/15B150 Safari/604.1

---

* HTTP Method: PATCH
* HTTP_USER_AGENT : Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_1 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Version/11.0 Mobile/15B150 Safari/604.1

Worth noting that from looking at the logs, the timing of almost all of them would seem to suggest that people probably aren't closing browsers or anything like that - something else is going on.

[elfassy]

something else is going on

Likely they are using the back button

…

On Sat, Dec 2, 2017, 23:26 John McDowall ***@***.***> wrote: Can confirm that this is still an issue on Rails 5.1.4. Deployed an application for the first time on Friday, and have received 7 ActionController::InvalidAuthenticityToken exception reports so far. Most seem to be webkit based mobile browsers (some iOS, some Android), but there are a couple of Deskop Mozilla ones in there: * HTTP Method: POST * HTTP_USER_AGENT : Mozilla/5.0 (Linux; Android 8.1.0; Nexus 5X Build/OPP6.171019.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.84 Mobile Safari/537.36 --- * HTTP Method: POST * HTTP_USER_AGENT : Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:57.0) Gecko/20100101 Firefox/57.0 --- * HTTP Method: POST * HTTP_USER_AGENT : Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 XL Build/OPD3.170816.023) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.71 Mobile Safari/537.36 --- * HTTP Method: POST * HTTP_USER_AGENT : Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_1 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Version/11.0 Mobile/15B150 Safari/604.1 --- * HTTP Method: POST * HTTP_USER_AGENT : Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.98 Safari/537.36 --- * HTTP Method: PATCH * HTTP_USER_AGENT : Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_1 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Version/11.0 Mobile/15B150 Safari/604.1 --- * HTTP Method: PATCH * HTTP_USER_AGENT : Mozilla/5.0 (iPhone; CPU iPhone OS 11_1_1 like Mac OS X) AppleWebKit/604.3.5 (KHTML, like Gecko) Version/11.0 Mobile/15B150 Safari/604.1 Worth noting that from looking at the logs, the timing of almost all of them would seem to suggest that people probably *aren't* closing browsers or anything like that - something else is going on. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#21948 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAS_77dk3-B7v7GO-QSkp3HOnqQbqWsNks5s8iLfgaJpZM4GNsnP> .

[rails-bot]

This issue has been automatically marked as stale because it has not been commented on for at least three months.
The resources of the Rails team are limited, and so we are asking for your help.
If you can still reproduce this error on the 5-1-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.

[clemens]

Still happening in 5.1.5. Haven't checked against master/5.2.

[rudolfolah]

I've been having this issue with devise signin; not sure what's going on, locally everything works fine but on my staging server login doesn't work.

update: the fix I found was to change the protect_from_forgery call to this in the application controller:

protect_from_forgery prepend: true

[BenFenner]

At our Rails shop we have a handful of web applications of various setups, all updated to Rails 5.1.6 and using protect_from_forgery with: exception in application_controller.rb. I think most are using cookie session store right now.
All of our applications are suffering from this issue.

I believe I have found a more generic, easily reproducible version of this bug (applicable to sites with login pages/session-tracked users). This doesn't seem to be browser or technology specific in any way. It doesn't require closing the browser either.

Steps to reproduce:

• Open two browser tabs and navigate both to the application login page. (form protected with CSRF)
• Log into the application using the first tab.
• Log out of the application using the first tab. (destroys the session)
• Attempt to log into application again using the second tab. (stale CSRF token can't match with non-existent session)

You will be presented with the InvalidAuthenticityToken error in the original post.

Interestingly enough, trying this for GitHub itself (ignoring the warning at the top about needing to refresh the page) seems to invoke the error, but the user is simply shown a static page, appearing as if the login attempt is taking a very long time to complete. The login attempt never completes.

[mindaslab]

I can confirm this, my app is breaking in Chrome while used on production. I use Rails 5.2

[Spone]

Same as @rudolfolah, I had to use protect_from_forgery with: :exception, prepend: true to fix the issue.

This new behavior is present since Rails 5.0 and has been added by this commit: 3979403

See also Devise README: https://github.com/plataformatec/devise/blob/715192a7709a4c02127afb067e66230061b82cf2/README.md#controller-filters-and-helpers

[lucascaton]

I can confirm that it still happens with a brand new app:

• Ruby 2.5.1
• Rails 5.2.1
• Safari 11.1.2 (13605.3.8)
• macOS 10.13.6 (High Sierra)

[lucascaton]

hi @maclover7 - can you please remove the needs feedback label?

[rails-bot]

This issue has been automatically marked as stale because it has not been commented on for at least three months.
The resources of the Rails team are limited, and so we are asking for your help.
If you can still reproduce this error on the 5-2-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.