Disallow raw SQL in dangerous AR methods #27947

btoews · 2017-02-08T18:47:59Z

Some AR methods allow raw SQL where raw SQL is rarely necessary. This opens up applications to SQL injection vulnerabilities when a developer unwittingly passes params or a model attribute to the AR method.

Code like this looks innocuous, though it actually is an exploitable SQL injection vulnerability:

class PostsController < ApplicationController
  def index
    @posts = Posts.order(params[:order_by]).limit(10)
  end
end

A probably-not-comprehensive list of such dangerous APIs can be found at http://rails-sqli.org/.

These APIs should be changed to not allow raw SQL. In the rare case where raw SQL is needed, the developer should be forced to acknowledge the risk by using an API whose name indicates the danger.

This would be a breaking change though, which I'm assuming would not be acceptable. This PR adds an AR config flag that, when enabled, restricts the arguments that can be passed to previously dangerous AR APIs. The previous functionality can be accessed using unsafe_raw_ prefixed APIs.

These changes take a bit of work, so I'm opening this PR early for discussion. So far, I've made changes to the #pluck, #order and #reorder APIs.

I'll be curious to hear what folks think about the idea. I'm happy to try a different approach if anyone has suggestions, but it would be really good to remove some sharp edges from AR.

rails-bot · 2017-02-08T18:48:04Z

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

eileencodes · 2017-02-08T18:50:26Z

cc/ @rails/security

eileencodes

I am 👍 on improving security to avoid SQL injection. I've left some comments about method names and a couple other things.

Thanks for working on this!

eileencodes · 2017-02-08T19:00:02Z

activerecord/lib/active_record/core.rb

+      # rename methods like #order to #unsafe_raw_order and will validate
+      # arguments to #order. A value of false will do nothing.
+      mattr_accessor :guard_unsafe_raw_sql, instance_writer: false
+      self.guard_unsafe_raw_sql = false


I think this would be better as allow_unsafe_raw_sql so that it's more likely to motivate someone to use it.

Agree it should be opt-out by using the unsafe_* API.

However, for compatibility with existing code, would expect to be able to turn off checks on a model superclass: SomeEngine::Model.guard_unsafe_raw_sql = false without affecting app models or other engines.

eileencodes · 2017-02-08T19:01:51Z

activerecord/lib/active_record/attribute_methods.rb

+      #
+      #   Person.effective_column_names
+      #   # => ["id", "created_at", "updated_at", "name", "age"]
+      def effective_column_names


I think this method name is doesn't really explain what it's doing, but I haven't been able to come up with a better name yet. safe_column_names? attribute_column_names?

attribute_names_and_aliases?

eileencodes · 2017-02-08T19:02:46Z

activerecord/lib/active_record/relation/calculations.rb

+      if has_include?(column_names.first)
+        construct_relation_for_association_calculations.pluck(*column_names)
+      else
+        if ActiveRecord::Base.guard_unsafe_raw_sql && !@klass.effective_column_name?(column_names)


I'd consider moving this into a method of it's own column_valid? or something like that.

eileencodes · 2017-02-08T19:04:06Z

activerecord/lib/active_record/relation/calculations.rb

+      end
+    end
+
+    def unsafe_raw_pluck(*column_names)


Should this have documentation?

jeremy

Cool!

jeremy · 2017-02-09T05:56:27Z

activerecord/lib/active_record/attribute_methods.rb

+      #
+      #   Person.effective_column_names
+      #   # => ["id", "created_at", "updated_at", "name", "age"]
+      def effective_column_names


attribute_names_and_aliases?

jeremy · 2017-02-09T05:58:22Z

activerecord/lib/active_record/attribute_methods.rb

+      def effective_column_names
+        @effective_column_names ||= begin
+          (attribute_names + attribute_aliases.keys).uniq
+        end


Can drop the begin/end

jeremy · 2017-02-09T06:10:06Z

activerecord/lib/active_record/attribute_methods.rb

+        end
+      end
+
+      def effective_column_name?(*names)


"Can I ask this model for attributes with these names?"

Person.respond_to_attributes? :foo, :bar

jeremy · 2017-02-09T06:15:50Z

activerecord/lib/active_record/attribute_methods.rb

+      end
+
+      def effective_column_name?(*names)
+        (names.flatten.map(&:to_s) - effective_column_names).empty?


When names is typically small, would expect that iterating through those and checking whether they're attributes would be quicker than subtracting each attr: names.all? { |name| has_attribute?(name) || has_attribute_alias?(name) }

(That'd obviate the effective_column_names method, too)

jeremy · 2017-02-09T06:18:04Z

activerecord/lib/active_record/relation/calculations.rb

+        return records.pluck(*column_names)
+      end
+
+      if has_include?(column_names.first)


Can drop the early return above and make this an elsif

jeremy · 2017-02-09T06:36:32Z

activerecord/lib/active_record/relation/query_methods.rb

+        hash = args.last.is_a?(Hash) ? args.pop : {}
+        directions.concat(hash.values)
+        columns.concat(hash.keys)
+        columns.concat(args)


Can do the assignments above here:

orderings = args.extract_options! attribute_names = args | orderings.keys

jeremy · 2017-02-09T06:37:00Z

activerecord/lib/active_record/relation/query_methods.rb

+          raise ArgumentError, "Invalid order column: #{bad_columns}"
+        end
+
+        bad_directions = directions.map(&:to_s) - %w(asc desc ASC DESC)


Rather than assign directions above, it's clear to ask for orderings.values here.

jeremy · 2017-02-09T06:39:44Z

activerecord/lib/active_record/core.rb

@@ -98,6 +98,13 @@ def self.error_on_ignored_order_or_limit=(value)
        self.error_on_ignored_order = value
      end

+      # :singleton-method:
+      # Specify the behavior for unsafe raw query methods. A value of true will


I get "unsafe," but how are these "raw?"

I went with unsafe_raw because 1) they are unsafe and 2) they allow raw SQL. I'd be happy to drop the raw bit if you'd like.

jeremy · 2017-02-09T06:40:29Z

activerecord/test/cases/unsafe_raw_sql_test.rb

+  ensure
+    ActiveRecord::Base.guard_unsafe_raw_sql = old_value
+  end
+end


Nice.

Coverage for the new unsafe_* API, too?

jeremy · 2017-02-09T06:54:06Z

activerecord/lib/active_record/core.rb

+      # rename methods like #order to #unsafe_raw_order and will validate
+      # arguments to #order. A value of false will do nothing.
+      mattr_accessor :guard_unsafe_raw_sql, instance_writer: false
+      self.guard_unsafe_raw_sql = false


Agree it should be opt-out by using the unsafe_* API.

However, for compatibility with existing code, would expect to be able to turn off checks on a model superclass: SomeEngine::Model.guard_unsafe_raw_sql = false without affecting app models or other engines.

rename config option. fix tests. s/effective_column_names/attribute_names_and_aliases/ s/effective_column_name?/attribute_name_or_alias?/ move pluck column checking logic into its own method document unsafe_raw methods 💄 s/attribute_name_or_alias?/respond_to_attributes?/ optimize respond_to_attributes? remove attribute_names_and_aliases 💄 typo dedup pluck/unsafe_raw_pluck logic with private method only call restrict_order_args unless unsafe-raw enabled 💄 refactor a bit typo test unsafe_raw methods too (and fix them)

activerecord/lib/active_record/relation/calculations.rb

btoews · 2017-02-09T17:04:42Z

Breaks ordering by a SQL expression. That's fine, but the only workaround is to disable checks globally. Would expect to selectively opt out of checks.

Similar, need an allowlist of ordering modifiers, probably delegated to Arel or the conn adapter. For example, Oracle ORDER BY foo NULLS FIRST.

Good call. I should have known there would be other crazy syntaxes 😄. I'll see if arel has anything to help with this.

btoews · 2017-02-09T17:37:42Z

I couldn't find any existing list of order modifiers that included NULLS FIRST/LAST. There is a VALID_DIRECTIONS constant, but it only includes ASC, and DESC. Should I just special case NULLS FIRST/LAST, or are you aware of other modifiers that should be included?

jeremy · 2017-02-09T19:30:14Z

Paraphrasing via @matthewd: We could use Arel.sql(…) to provide intentional SQL literals for "unsafe" cases. Then we don't split safe/unsafe API, don't need to pass special flags around, and have a clear compatibility path.

Then we could do a few modes for unsafe query checking: strict (new apps, raise on violations), deprecated (default, warn on violations that didn't use SQL literals), false (off, shush).

activerecord/lib/active_record/attribute_methods.rb

activerecord/lib/active_record/querying.rb

activerecord/lib/active_record/relation/calculations.rb

jeremy · 2017-02-09T19:37:37Z

activerecord/lib/active_record/relation/calculations.rb

+          records.pluck(*column_names)
+        elsif has_include?(column_names.first)
+          construct_relation_for_association_calculations.pluck(*column_names)
+        elsif unsafe_raw || unrecognized.none?


Feels like this is a conditional checking the guard condition, but it's actually checking whether not to guard, leaving the raise in the else disconnected from the decisionmaking that leads to it. Compare with:

elsif !allow_unsafe_raw && unrecognized.any? raise … else # actual pluck impl end

jeremy · 2017-02-09T19:38:41Z

activerecord/lib/active_record/relation/query_methods.rb

+          raise ArgumentError, "Invalid order direction: #{unrecognized}"
+        end
+      end
+


jeremy · 2017-02-09T19:39:23Z

activerecord/lib/active_record/relation/query_methods.rb

+        # TODO: find a better list of modifiers.
+        unrecognized = orderings.values.reject { |d| VALID_DIRECTIONS.include?(d.to_s) }
+        if unrecognized.any?
+          raise ArgumentError, "Invalid order direction: #{unrecognized}"


Consider unrecognized.inspect so the message reflects the string/symbol syntax from the caller.

jeremy · 2017-02-09T19:41:54Z

I couldn't find any existing list of order modifiers that included NULLS FIRST/LAST. There is a VALID_DIRECTIONS constant, but it only includes ASC, and DESC. Should I just special case NULLS FIRST/LAST, or are you aware of other modifiers that should be included?

Think rolling with VALID_DIRECTIONS is the way to go since we already have it. Introducing other ordering modifiers is a separate effort.

btoews · 2017-02-09T19:55:04Z

We could use Arel.sql(…) to provide intentional SQL literals for "unsafe" cases. Then we don't split safe/unsafe API, don't need to pass special flags around, and have a clear compatibility path.

My inexperience working with AR is showing: I don't follow that statement at all 😄.

jeremy · 2017-02-09T21:14:06Z

My bad 😊

Boils down to this:

>> Arel.sql('foo')
=> "foo"
>> Arel.sql('foo').class
=> Arel::Nodes::SqlLiteral

If we get an ordering that's an Arel::Nodes::SqlLiteral, then we skip the guard. Otherwise, we enforce the guard.

Example

unrecognized = column_names.reject do |cn|
  cn.is_a?(Arel::Nodes::SqlLiteral) || @klass.respond_to_attribute?(cn)
end

and

unrecognized = orderings.values.reject do |d|
  d.is_a?(Arel::Nodes::SqlLiteral) || VALID_DIRECTIONS.include?(d)
end

To use unsafe raw SQL, you'd do .order(Arel.sql('raw stuff')) and .pluck(Arel.sql(…)) instead of introducing a new set of .unsafe_raw_*('raw stuff') APIs.

Finally, to do a deprecation to encourage moves to Arel.sql style without breaking existing code, we could insert warnings like

unrecognized = orderings.values.reject do |d|
  d.is_a?(Arel::Nodes::SqlLiteral) || VALID_DIRECTIONS.include?(d) ||
    (@klass.allow_unsafe_raw_sql == :deprecated).tap do |warn|
      ActiveSupport::Deprecation.warn "Ordering in the #{d.inspect} direction is unrecognized and considered unsafe, vulnerable to SQL injection. Switch to `Arel.sql(#{d.to_s.inspect})` if this is intentional custom SQL ordering." if warn
    end
end

matthewd · 2017-02-09T23:17:04Z

If we get an ordering that's an Arel::Nodes::SqlLiteral, then we skip the guard.

Tiny logic tweak: we should just let any Arel node through

t27duck · 2017-02-12T13:53:37Z

Sorry to interrupt, but I feel like this is putting the cart before the horse. It's relative common to pass raw SQL snippets into order as there is no other way the method allows developers to specify other tables in the ORDER BY.

Example use case:

class Game < ActiveRecord::Base
  # id - integer
  # platform_id - integer
  # title - string
  # ...
  belongs_to :platform
end

class Platform < ActiveRecord::Base
  # id - integer
  # name - string
  # ...
  has_many :games
end

If we want to display a table of games sorted by their platform, we have to do this in our code:

@games = Game.joins(:platform).order("#{Platform.quoted_table_name}.name ASC")

While I'm pretty sure most developers would love to instead be able to do this (similar to how where handles joined tables:

@games = Game.joins(:platform).order(platforms: {name: :asc})

Even if the change is to allow all Arel::Nodes::SqlLiteral objects passed into order it would still require some rework of existing code due to the limitation of the current API.

matthewd · 2017-02-12T14:26:55Z

Even if the change is to allow all Arel::Nodes::SqlLiteral objects passed into order it would still require some rework of existing code

@t27duck I'm not sure what you're suggesting. Surely any change that addresses the core problem (that the method accepts raw SQL, and that callers might reasonably not account for that) is going to require some change to existing callers?

t27duck · 2017-02-12T14:44:59Z

Point taken about any changes to the public API would require developers to adjust. Perhaps I'm not 100% sold on blocking out raw strings to the interface.

What if, instead, ActionController::Parameters aren't allowed in? Is there a way to ask if params[:order_by] came from a request?

To your suggestion on allowing Arel::Nodes::SqlLiteral to be passed ... I don't think it really prevents anything as a developer can do this to bypass this proposed check:

@posts = Post.order(Arel::Notes::SqlLiteral.new(params[:order_by]))

All it would take is one person asking on stackoverflow "Why I can't pass a string into order" and one poorly thought out answer to suggest this "workaround".

btoews · 2017-02-14T18:42:49Z

What if, instead, ActionController::Parameters aren't allowed in? Is there a way to ask if params[:order_by] came from a request?

We need to protect against model attributes in addition to parameters.

All it would take is one person asking on stackoverflow "Why I can't pass a string into order" and one poorly thought out answer to suggest this "workaround".

This is why I'd rather prefix dangerous method names with unsafe_raw_. I'm happy to defer that decision to core folks though.

jeremy · 2017-02-14T23:47:12Z

All it would take is one person asking on stackoverflow "Why I can't pass a string into order" and one poorly thought out answer to suggest this "workaround".

Same for bypassing strong params, HTML safety, and any other security measure. "Why can't I pass a string to #order?" → "Oh just change that to #unsafe_raw_order." Of course there is a spectrum from "cuts hand on contact" to "can shoot self in foot" to "impossible to use in error," so we need to keep our heads on straight about where we land on that spectrum and why.

Foremost, provide a fluent, sensible, obvious interface that Rails developers are happy to use and is secure in default, typical usage.

Secondarily, make it awkward for users to inadvertently construct a foot-gun from otherwise-safe tools.

People copy/pasting senseless API usage that bypasses the secure default? These are not the developers we want to hedge against when crafting our API.

In this case, exposing a single, safe set of relation builder methods is a great way to tackle both. Arguments are allowlisted by default, which catches inadvertent injection vulns, but that can be explicitly bypassed with raw SQL when needed. That's right in the sweet spot of the Active Record way: safe abstraction plus a moderately-vinegared bypass.

- Followup of rails/rails#27947 - Silences bunch of deprecation warnings.

matthewd · 2018-09-25T13:18:04Z

@harshal0608 this change adds an extra confirmation of intent for reorder, but as discussed above, that doesn't apply for find_by and where: they are operating as intended, and can't be "fixed" any more than eval can.

Rails 5.2 adds deprecation warnings for several forms of SQL in ActiveRecord queries, which Rails 6 will no longer allow. As part of the ongoing work to upgrade to Rails 6, we want to address these warnings. Mostly, this involves wrapping SQL calls which we know are safe (ie, calls that don't reference any user-editable model fields) in `Arel.sql`; in some cases, we're also able to replace the SQL with built-in ActiveRecord methods. - [Rails 5.2 stops some raw SQL, prevents SQL injections](https://blog.bigbinary.com/2018/10/16/rails-5-2-disallows-raw-sql-in-active-record.html) - [Disallow raw SQL in dangerous AR methods](rails/rails#27947)

rails#27947 was intended to address `Post.order(params[:order])` which leading to SQL injection. But the protection caused very annoying warnings, especially for calling by a string literal which invoke a function like `Post.order("length(title)")`. Calling by a string literal should be safe since it is not an user input. So I'd like to allow string literals as a safe raw SQL string. There is no reliable way to distinguish between `params[:order]` and `"length(title)"`, but at least `params[:order]` as an user input is always a mutable string, and `"length(title)"` will be a fstring if `# frozen_string_literal: true`. So I think that we can add fstring to the allowlist to mitigate the annoying warnings without losing the original intention of rails#27947. Resolves rails#32995.

Raw SQL passed to `pluck` now has to be explicitly marked as SQL via Arel.sql, see rails/rails#27947

mastahyeti added 2 commits February 8, 2017 11:23

add config to check arguments to unsafe AR methods

fda8480

same for #order and #reorder

30ecd74

rails-bot assigned eileencodes Feb 8, 2017

eileencodes added activerecord security labels Feb 8, 2017

eileencodes requested changes Feb 8, 2017

View reviewed changes

maclover7 added the needs work label Feb 8, 2017

jeremy reviewed Feb 9, 2017

View reviewed changes

btoews commented Feb 9, 2017

View reviewed changes

activerecord/lib/active_record/relation/calculations.rb Outdated Show resolved Hide resolved

docs for reorder methods too

2a30621

use VALID_DIRECTIONS constant

e2bdd70

jeremy reviewed Feb 9, 2017

View reviewed changes

btoews added 3 commits February 21, 2017 10:50

change config to enabled/deprecated/disabled

4d58a43

allow Arel.sql() for pluck

1fd4108

allow Arel.sql(...) for order/reorder

3613e2c

kamipo mentioned this pull request Jul 10, 2018

Allow fstring as a safe raw SQL string #33330

Closed

prathamesh-sonpatki added a commit to codetriage/CodeTriage that referenced this pull request Jul 27, 2018

Disallow Raw SQL by using Arel.sql for the raw part of the query

79018e1

- Followup of rails/rails#27947 - Silences bunch of deprecation warnings.

prathamesh-sonpatki mentioned this pull request Jul 27, 2018

Disallow Raw SQL by using Arel.sql for the raw part of the query codetriage/CodeTriage#725

Merged

prathamesh-sonpatki added a commit to codetriage/CodeTriage that referenced this pull request Jul 27, 2018

Disallow Raw SQL by using Arel.sql for the raw part of the query (#725)

a051222

- Followup of rails/rails#27947 - Silences bunch of deprecation warnings.

robin850 mentioned this pull request Oct 28, 2018

It's not possible to chain #left_joins just like #joins method #34332

Closed

jasnow mentioned this pull request Jun 10, 2019

Upgrade Rails to 5.2 or 6.0 and Ruby to 2.7.0-preview1 OWASP/railsgoat#364

Closed

stgm mentioned this pull request Dec 28, 2019

Make warning/error more explicit for raw SQL arguments to query methods #38092

Closed

ankane mentioned this pull request Sep 5, 2020

Deprecate raw SQL in group method without marking the SQL as safe #40183

Closed

Hamms mentioned this pull request Nov 3, 2020

Address unsafe SQL in ActiveRecord queries code-dot-org/code-dot-org#37654

Closed

8 tasks

ClearlyClaire mentioned this pull request Mar 25, 2021

Fix /admin/tags/:id crashing since Rails 6.1 update mastodon/mastodon#15953

Merged

ClearlyClaire added a commit to ClearlyClaire/mastodon that referenced this pull request Mar 25, 2021

Fix /admin/tags/:id crashing since Rails 6.1 update

1f5a4d9

Raw SQL passed to `pluck` now has to be explicitly marked as SQL via Arel.sql, see rails/rails#27947

Gargron pushed a commit to mastodon/mastodon that referenced this pull request Mar 26, 2021

Fix /admin/tags/:id crashing since Rails 6.1 update (#15953)

a650a11

Raw SQL passed to `pluck` now has to be explicitly marked as SQL via Arel.sql, see rails/rails#27947

glasses618 mentioned this pull request Oct 16, 2021

Wrapping raw SQL in Arel.sql() khiav223577/find_with_order#24

Merged

chrisguida pushed a commit to Start9Labs/mastodon that referenced this pull request Feb 26, 2022

Fix /admin/tags/:id crashing since Rails 6.1 update (mastodon#15953)

655011f

Raw SQL passed to `pluck` now has to be explicitly marked as SQL via Arel.sql, see rails/rails#27947

brendanshean mentioned this pull request Jun 21, 2022

Wrap query methods in Arel.sql empirical-org/Empirical-Core#9288

Merged

Disallow raw SQL in dangerous AR methods #27947

Disallow raw SQL in dangerous AR methods #27947

Conversation

btoews commented Feb 8, 2017

rails-bot commented Feb 8, 2017

eileencodes commented Feb 8, 2017

eileencodes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

btoews commented Feb 9, 2017

btoews commented Feb 9, 2017

jeremy commented Feb 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremy commented Feb 9, 2017

btoews commented Feb 9, 2017

jeremy commented Feb 9, 2017

matthewd commented Feb 9, 2017

t27duck commented Feb 12, 2017

matthewd commented Feb 12, 2017

t27duck commented Feb 12, 2017

btoews commented Feb 14, 2017

jeremy commented Feb 14, 2017

matthewd commented Sep 25, 2018