Add inspection_masks to make values of sensitive database columns w…

[piecehealth]

…on't be exposed while call #inspect.

• Why:
  Some sensitive data will be exposed in log accidentally, e.g.

account = Account.find params[:id]
payload = { account: @account }
logger.info "payload will be #{ payload }"

All the information of account will be exposed in log.

• Solution:
  Add a class attribute inspection_masks to specify which values of columns shouldn't be exposed.

class Account < ActiveRecord::Base
  self.inspection_masks = [:credit_card_number]
end

Account.last.insepct # => #<Account id: 123, credit_card_number: [FILTERED] ...>

Summary

Provide a general description of the code changes in your pull
request... were there any bugs you had fixed? If so, mention them. If
these bugs have open GitHub issues, be sure to tag them here as well,
to keep the conversation linked together.

Other Information

If there's anything else that's important and relevant to your pull
request, mention that information here. This could include
benchmarks, or other information.

If you are updating any of the CHANGELOG files or are asked to update the
CHANGELOG files by reviewers, please add the CHANGELOG entry at the top of the file.

Finally, if your pull request affects documentation or any non-code
changes, guidelines for those changes are available
here

Thanks for contributing to Rails!

[rafaelfranca]

Don't need to freeze this string.

[piecehealth]

Hi @rafaelfranca , I have removed freeze per your comment, please take a look, thanks.

[eileencodes]

This is great! I'm wondering though, would it be better to piggyback on the config.filter_parameters option and by default always filter those in inspect? I'd imagine you wouldn't want credit card number in inspect or in the logs, so it makes sense to me to use the same list. Thoughts?

[piecehealth]

Hi @eileencodes , thanks for your advise, I checked in a new commit based on your comment.

I don't know how to write test about railtie part, but I do test on my local 😉

[piecehealth]

trigger CI

[eileencodes]

@piecehealth you should be able to add a test to railties/test/application/configuration_test.rb that asserts that inspection_masks are equivalent to filter_parameters.

I'm not sure if inspection_masks is my favorite phrasing. I'm thinking it should match the filter_parameters and be called filter_attributes. I need a second opinion though. @rafaelfranca what do you think?

[matthewd]

I'm not sure if inspection_masks is my favorite phrasing. I'm thinking it should match the filter_parameters and be called filter_attributes.

I agree it'd be better to have them sound more related. My only worry about filter_attributes in particular would be that it could also describe ignored_columns... OTOH, the same could be said about filter_parameters vs strong params. ¯\_(ツ)_/¯

We definitely also need to cover pretty_print below, which is basically another more complicated inspect method. But beyond that, the other place we show attribute values is SQL logs; we can't do much about it if prepared statements is off, but at least when it's on, we could suppress the values.

Finally, on a behavioural note: I suggest we do not mask the value if it's nil (or maybe blank?).

[piecehealth]

Thanks @eileencodes @matthewd for your review, I have made some changes according your comments.

• Change class variable name: inspection_masks -> filter_attributes.
• Add test case for railtie.
• Don't mask nil value.
• Also support pretty_print.

By the way, it's really hard for non English speaker to name variable 😢

[eugeneius]

Since this will have include? called on it for each attribute in the model, I think we should convert it to a set (include? is linear time for an array, but constant time for a set).

It might also be worth doing the conversion on assignment, as we do for ignored_columns:

rails/activerecord/lib/active_record/model_schema.rb

Lines 274 to 276 in e6ba30e

	def ignored_columns=(columns)
	@ignored_columns = columns.map(&:to_s)
	end

[eugeneius]

This is unnecessary, and in fact it causes a problem: if the user sets filter_attributes in an initializer, their configuration will be overwritten by filter_parameters and lost.

It might be worth adding a test for this case.

[piecehealth]

Thanks for your valuable advice, I just wonder is it necessary for user to set filter_attributes in an initializer? Since user could set filter_attributes in ApplicationRecord e.g.

class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true
  self.filter_attributes = [:credit_card_number]
end

Otherwise, I should add a new instance variable @filter_attributes in Rails::Application::Configuration, please correct me if there is any mistake, thanks 😉

[eugeneius]

Any option on ActiveRecord::Base can be configured by setting a key on Rails.application.config.active_record. I was thinking of someone adding an initializer that mirrors the one for filter_parameters, since they're so similar:

https://github.com/rails/rails/blob/e6ba30efbf107954db2707410d78d77680d69995/railties/lib/rails/generators/rails/app/templates/config/initializers/filter_parameter_logging.rb.tt

[eugeneius]

These hooks are run in the context of ActiveRecord::Base, and it's idiomatic to use self instead of referring to the constant explicitly, e.g:

rails/activerecord/lib/active_record/railtie.rb

Lines 135 to 137 in e6ba30e

	initializer "active_record.initialize_database" do
	ActiveSupport.on_load(:active_record) do
	self.configurations = Rails.application.config.database_configuration

[rafaelfranca]

Awesome! Very good patch. Can you squash your commit?

[piecehealth]

@rafaelfranca @eileencodes thanks for your time, I have squashed my commits and the CI is passed 😃

[eliotsykes]

Pleased to see this, lovely addition 👍

Currently it will behave inconsistently for apps configured with regex or callable filter_parameters.

For those apps, affected request parameters successfully filtered in the logs will not be filtered in #inspect.

Related examples from actionpack/lib/action_dispatch/http/filter_parameters.rb docs:

    # Allows you to specify sensitive parameters which will be replaced from
    # the request log by looking in the query string of the request and all
    # sub-hashes of the params hash to filter. Filtering only certain sub-keys
    # from a hash is possible by using the dot notation: 'credit_card.number'.
    # If a block is given, each key and value of the params hash and all
    # sub-hashes are passed to it, where the value or the key can be replaced using
    # String#replace or similar methods.
    #
    #   env["action_dispatch.parameter_filter"] = [:password]
    #   => replaces the value to all keys matching /password/i with "[FILTERED]"
    #
    #   env["action_dispatch.parameter_filter"] = [:foo, "bar"]
    #   => replaces the value to all keys matching /foo|bar/i with "[FILTERED]"
    #
    #   env["action_dispatch.parameter_filter"] = [ "credit_card.code" ]
    #   => replaces { credit_card: {code: "xxxx"} } with "[FILTERED]", does not
    #   change { file: { code: "xxxx"} }
    #
    #   env["action_dispatch.parameter_filter"] = -> (k, v) do
    #     v.reverse! if k =~ /secret/i
    #   end
    #   => reverses the value to all keys matching /secret/i

[reckerswartz]

can this be used or improved for
the issue on devise to prevent password reset tokens leak?
https://github.com/plataformatec/devise#password-reset-tokens-and-rails-logs

[brentdodell]

This is great! I'm wondering though, would it be better to piggyback on the config.filter_parameters option and by default always filter those in inspect? I'd imagine you wouldn't want credit card number in inspect or in the logs, so it makes sense to me to use the same list. Thoughts?

I saw that @eileencodes mentioned this above. Has there been any further discussion about combining the two in a future update? It seems to me that sensitive data is sensitive data and trying to maintain two lists is likely to be problematic. Nevermind, I see that filter_parameters are appended to filter_attributes.

Also, is there any chance that this will make it into a security patch for 5.2?

[eileencodes]

Also, is there any chance that this will make it into a security patch for 5.2?

Nope, we don't backport features even when they're security related. Security patches are only for vulnerabilities.