Disallow raw SQL in dangerous AR methods

[mastahyeti]

Some AR methods allow raw SQL where raw SQL is rarely necessary. This opens up applications to SQL injection vulnerabilities when a developer unwittingly passes params or a model attribute to the AR method.

Code like this looks innocuous, though it actually is an exploitable SQL injection vulnerability:

class PostsController < ApplicationController
  def index
    @posts = Posts.order(params[:order_by]).limit(10)
  end
end

A probably-not-comprehensive list of such dangerous APIs can be found at http://rails-sqli.org/.

These APIs should be changed to not allow raw SQL. In the rare case where raw SQL is needed, the developer should be forced to acknowledge the risk by using an API whose name indicates the danger.

This would be a breaking change though, which I'm assuming would not be acceptable. This PR adds an AR config flag that, when enabled, restricts the arguments that can be passed to previously dangerous AR APIs. The previous functionality can be accessed using unsafe_raw_ prefixed APIs.

These changes take a bit of work, so I'm opening this PR early for discussion. So far, I've made changes to the #pluck, #order and #reorder APIs.

I'll be curious to hear what folks think about the idea. I'm happy to try a different approach if anyone has suggestions, but it would be really good to remove some sharp edges from AR.

[rails-bot]

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

This repository is being automatically checked for code quality issues using Code Climate. You can see results for this analysis in the PR status below. Newly introduced issues should be fixed before a Pull Request is considered ready to review.

Please see the contribution instructions for more information.

[eileencodes]

cc/ @rails/security

[eileencodes]

I am 👍 on improving security to avoid SQL injection. I've left some comments about method names and a couple other things.

Thanks for working on this!

[eileencodes]

I think this would be better as allow_unsafe_raw_sql so that it's more likely to motivate someone to use it.

[jeremy]

Agree it should be opt-out by using the unsafe_* API.

However, for compatibility with existing code, would expect to be able to turn off checks on a model superclass: SomeEngine::Model.guard_unsafe_raw_sql = false without affecting app models or other engines.

[eileencodes]

I think this method name is doesn't really explain what it's doing, but I haven't been able to come up with a better name yet. safe_column_names? attribute_column_names?

[jeremy]

attribute_names_and_aliases?

[eileencodes]

I'd consider moving this into a method of it's own column_valid? or something like that.

[eileencodes]

Should this have documentation?

[jeremy]

Cool!

[jeremy]

attribute_names_and_aliases?

[jeremy]

Can drop the begin/end

[jeremy]

"Can I ask this model for attributes with these names?"

Person.respond_to_attributes? :foo, :bar

[jeremy]

When names is typically small, would expect that iterating through those and checking whether they're attributes would be quicker than subtracting each attr: names.all? { |name| has_attribute?(name) || has_attribute_alias?(name) }

[jeremy]

(That'd obviate the effective_column_names method, too)

[jeremy]

Can drop the early return above and make this an elsif

[jeremy]

Can do the assignments above here:

orderings = args.extract_options!
attribute_names = args | orderings.keys

[jeremy]

Rather than assign directions above, it's clear to ask for orderings.values here.

[jeremy]

I get "unsafe," but how are these "raw?"

[mastahyeti]

I went with unsafe_raw because 1) they are unsafe and 2) they allow raw SQL. I'd be happy to drop the raw bit if you'd like.

[jeremy]

Nice.

Coverage for the new unsafe_* API, too?

[jeremy]

Agree it should be opt-out by using the unsafe_* API.

However, for compatibility with existing code, would expect to be able to turn off checks on a model superclass: SomeEngine::Model.guard_unsafe_raw_sql = false without affecting app models or other engines.

[mastahyeti]

Is the _ prefix okay here, or should I choose something more descriptive like #pluck_internal?:

[jeremy]

That's cool. We use single-_ elsewhere too.

[jeremy]

Would use a keyword arg for the allow_unsafe_raw: boolean.

[mastahyeti]

Breaks ordering by a SQL expression. That's fine, but the only workaround is to disable checks globally. Would expect to selectively opt out of checks.

Similar, need an allowlist of ordering modifiers, probably delegated to Arel or the conn adapter. For example, Oracle ORDER BY foo NULLS FIRST.

Good call. I should have known there would be other crazy syntaxes 😄. I'll see if arel has anything to help with this.

[mastahyeti]

I couldn't find any existing list of order modifiers that included NULLS FIRST/LAST. There is a VALID_DIRECTIONS constant, but it only includes ASC, and DESC. Should I just special case NULLS FIRST/LAST, or are you aware of other modifiers that should be included?

[jeremy]

Paraphrasing via @matthewd: We could use Arel.sql(…) to provide intentional SQL literals for "unsafe" cases. Then we don't split safe/unsafe API, don't need to pass special flags around, and have a clear compatibility path.

Then we could do a few modes for unsafe query checking: strict (new apps, raise on violations), deprecated (default, warn on violations that didn't use SQL literals), false (off, shush).

[jeremy]

attribute_types.key?(name) a tad quicker?

[jeremy]

No biggie, but hard to review diffs when line breaks change too.

[jeremy]

That's cool. We use single-_ elsewhere too.

[jeremy]

Would use a keyword arg for the allow_unsafe_raw: boolean.

[jeremy]

Feels like this is a conditional checking the guard condition, but it's actually checking whether not to guard, leaving the raise in the else disconnected from the decisionmaking that leads to it. Compare with:

elsif !allow_unsafe_raw && unrecognized.any?
  raise …
else
  # actual pluck impl
end

[jeremy]

👍

[jeremy]

Consider unrecognized.inspect so the message reflects the string/symbol syntax from the caller.

[jeremy]

I couldn't find any existing list of order modifiers that included NULLS FIRST/LAST. There is a VALID_DIRECTIONS constant, but it only includes ASC, and DESC. Should I just special case NULLS FIRST/LAST, or are you aware of other modifiers that should be included?

Think rolling with VALID_DIRECTIONS is the way to go since we already have it. Introducing other ordering modifiers is a separate effort.

[mastahyeti]

We could use Arel.sql(…) to provide intentional SQL literals for "unsafe" cases. Then we don't split safe/unsafe API, don't need to pass special flags around, and have a clear compatibility path.

My inexperience working with AR is showing: I don't follow that statement at all 😄.

[jeremy]

My bad 😊

Boils down to this:

>> Arel.sql('foo')
=> "foo"
>> Arel.sql('foo').class
=> Arel::Nodes::SqlLiteral

If we get an ordering that's an Arel::Nodes::SqlLiteral, then we skip the guard. Otherwise, we enforce the guard.

Example

unrecognized = column_names.reject do |cn|
  cn.is_a?(Arel::Nodes::SqlLiteral) || @klass.respond_to_attribute?(cn)
end

and

unrecognized = orderings.values.reject do |d|
  d.is_a?(Arel::Nodes::SqlLiteral) || VALID_DIRECTIONS.include?(d)
end

To use unsafe raw SQL, you'd do .order(Arel.sql('raw stuff')) and .pluck(Arel.sql(…)) instead of introducing a new set of .unsafe_raw_*('raw stuff') APIs.

Finally, to do a deprecation to encourage moves to Arel.sql style without breaking existing code, we could insert warnings like

unrecognized = orderings.values.reject do |d|
  d.is_a?(Arel::Nodes::SqlLiteral) || VALID_DIRECTIONS.include?(d) ||
    (@klass.allow_unsafe_raw_sql == :deprecated).tap do |warn|
      ActiveSupport::Deprecation.warn "Ordering in the #{d.inspect} direction is unrecognized and considered unsafe, vulnerable to SQL injection. Switch to `Arel.sql(#{d.to_s.inspect})` if this is intentional custom SQL ordering." if warn
    end
end

[matthewd]

If we get an ordering that's an Arel::Nodes::SqlLiteral, then we skip the guard.

Tiny logic tweak: we should just let any Arel node through

[t27duck]

Sorry to interrupt, but I feel like this is putting the cart before the horse. It's relative common to pass raw SQL snippets into order as there is no other way the method allows developers to specify other tables in the ORDER BY.

Example use case:

class Game < ActiveRecord::Base
  # id - integer
  # platform_id - integer
  # title - string
  # ...
  belongs_to :platform
end

class Platform < ActiveRecord::Base
  # id - integer
  # name - string
  # ...
  has_many :games
end

If we want to display a table of games sorted by their platform, we have to do this in our code:

@games = Game.joins(:platform).order("#{Platform.quoted_table_name}.name ASC")

While I'm pretty sure most developers would love to instead be able to do this (similar to how where handles joined tables:

@games = Game.joins(:platform).order(platforms: {name: :asc})

Even if the change is to allow all Arel::Nodes::SqlLiteral objects passed into order it would still require some rework of existing code due to the limitation of the current API.

[matthewd]

Even if the change is to allow all Arel::Nodes::SqlLiteral objects passed into order it would still require some rework of existing code

@t27duck I'm not sure what you're suggesting. Surely any change that addresses the core problem (that the method accepts raw SQL, and that callers might reasonably not account for that) is going to require some change to existing callers?

[t27duck]

Point taken about any changes to the public API would require developers to adjust. Perhaps I'm not 100% sold on blocking out raw strings to the interface.

What if, instead, ActionController::Parameters aren't allowed in? Is there a way to ask if params[:order_by] came from a request?

To your suggestion on allowing Arel::Nodes::SqlLiteral to be passed ... I don't think it really prevents anything as a developer can do this to bypass this proposed check:

@posts = Post.order(Arel::Notes::SqlLiteral.new(params[:order_by]))

All it would take is one person asking on stackoverflow "Why I can't pass a string into order" and one poorly thought out answer to suggest this "workaround".

[mastahyeti]

What if, instead, ActionController::Parameters aren't allowed in? Is there a way to ask if params[:order_by] came from a request?

We need to protect against model attributes in addition to parameters.

All it would take is one person asking on stackoverflow "Why I can't pass a string into order" and one poorly thought out answer to suggest this "workaround".

This is why I'd rather prefix dangerous method names with unsafe_raw_. I'm happy to defer that decision to core folks though.

[jeremy]

All it would take is one person asking on stackoverflow "Why I can't pass a string into order" and one poorly thought out answer to suggest this "workaround".

Same for bypassing strong params, HTML safety, and any other security measure. "Why can't I pass a string to #order?" → "Oh just change that to #unsafe_raw_order." Of course there is a spectrum from "cuts hand on contact" to "can shoot self in foot" to "impossible to use in error," so we need to keep our heads on straight about where we land on that spectrum and why.

Foremost, provide a fluent, sensible, obvious interface that Rails developers are happy to use and is secure in default, typical usage.

Secondarily, make it awkward for users to inadvertently construct a foot-gun from otherwise-safe tools.

People copy/pasting senseless API usage that bypasses the secure default? These are not the developers we want to hedge against when crafting our API.

In this case, exposing a single, safe set of relation builder methods is a great way to tackle both. Arguments are allowlisted by default, which catches inadvertent injection vulns, but that can be explicitly bypassed with raw SQL when needed. That's right in the sweet spot of the Active Record way: safe abstraction plus a moderately-vinegared bypass.

[mastahyeti]

Sorry to take so long with those last updates. I was on vacation. This branch should be good to go now?

[mastahyeti]

Bump. Is this ready to merge @matthewd?

[matthewd]

Yes, thanks 👍🏻

Sorry, I've been sitting on it because I feel reluctant to squash all of this work and history down to a single commit -- but the more I stare, the less I can see historically-relevant divisions to rebase it into a new series. 😕

[matthewd]

@mastahyeti so I had a go at a semi-squash that I think retains the core narrative of our design process, but cuts out the cross-merges and fixups: rails:master...matthewd:unsafe_raw_sql

Are you okay with that series? The commits still have your name on them, so even though they're just re-stitchings of your originals, I'd rather get your +1 before I merge.

[mastahyeti]

That series looks great @matthewd. Thanks for taking care of that.

[matthewd]

🎉 a1ee43d 🎉

Thanks again for all your work on this @mastahyeti -- I'm really excited about how much this will improve everyone's safety!

[mastahyeti]

Thanks for all the help getting this shipped!

[pschambacher]

Congrats to make our lives easier by having to worry less about sql injection.
Just to be sure, if we have code like this:

MyModel.pluck('distinct(`my_models`.`name`)')

Should I replace it with the following?

MyModel.pluck(Arel.sql('distinct(`my_models`.`name`)'))

I feel like I'm going to miss Squeel again.

[mastahyeti]

Should I replace it with the following?

Yes.

[matthewd]

@pschambacher or MyModel.distinct.pluck(:name)

[Fernan2]

It's pretty obvious that Post.order("length(title)") is safe, because it contains no user inputs, while Post.order(params[:my_order]) is not... but both are treated as unsafe. I wonder if that's a needed pain because there's no way to distinguish between them, or if treating Post.order("length(title)") as unsafe is really intended.

Also I'd like to know if Arel.sql() is the way to deal with NULLS LAST sorting...

[matthewd]

@harshal0608 this change adds an extra confirmation of intent for reorder, but as discussed above, that doesn't apply for find_by and where: they are operating as intended, and can't be "fixed" any more than eval can.