Ent Encryption

Certain industries have strong regulation around the storage of personal data. Private medical data, financial data, social security numbers, credit card numbers, all of these things are sensitive. Sidekiq Enterprise v1.3.0+ supports transparent encryption of the last job arguments (aka the secret bag) so sensitive data at rest in Redis cannot be seen.

Enable Encryption

In your initializer, activate the feature:

Sidekiq::Enterprise::Crypto.enable(active_version: 1) do |version|
  # this block should return the key for the version N
  #
  # every time you need to rotate the crypto key, you should bump
  # the active_version. this block should always return the key for 
  # the given version.
  #
  # You can store the key for each version in a file, in an ENV variable or on a remote
  # keystore - it's up to you how to do key management.
end

You can create a new random key in irb like so:

require 'openssl'
File.open("/var/crypto/secret.1.key", "w") { |file| file.write(OpenSSL::Cipher.new("aes-256-cbc").random_key) }

Configure your Private Workers

Sidekiq uses a special pattern to keep sensitive data private while also allowing some arguments to be cleartext to help with debugging job failures in the Web UI. The last argument for an encrypted worker is the secret bag and will be encrypted - all other arguments are cleartext to aid with debugging. I recommend passing a Hash of as the last argument but anything can be passed.

This has one special side effect: all encrypted workers must take >= 2 arguments. If you don't wish to have any cleartext, you can use perform(nil, secret_bag).

Tell Sidekiq to keep a Worker's arguments private:

class PrivateWorker
  include Sidekiq::Worker
  sidekiq_options encrypt: true

  def perform(x, y, secret_bag)
  end
end

Now create a new job with the secret bag as the last parameter:

SecretWorker.perform_async(1, 2, {"ssn" => "123-45-6789"})

Within Redis, the job data will look something like this:

{"class"=>"SecretWorker", "args"=>[1, 2, "BAhTOhFTaWRla2lxOjpFbmMIOgdpdiIVo1mbHmnVxiOITRFamysuBzoGdmkGOglibG9iIjUsYKydz73NLUB952KmES-K2LURDF6I_dWTBtzOX6493OaqNl6KZta19vWTKRwX9KQ="], "retry"=>true, "queue"=>"default", "encrypt"=>true, "jid"=>"b5f7df591b368910db3fc2d2", "created_at"=>1468597911.397162, "enqueued_at"=>1468597911.3972108}

Notes

• The encryption header adds about 100 bytes to the size of arguments plus 30% Base64 encoding overhead. Arguments which are 1000 bytes in plaintext are about 1400 bytes when encrypted. The Base64 encoding is necessary as JSON content must be valid UTF8 and the encrypted data is binary.
• ONLY the last argument is encrypted. Any error message and backtrace will still be plaintext within a job. Be careful not to expose any sensitive data when raising errors.
• The unique jobs feature will not work on encrypted jobs, since the encrypted argument is always unique. If a Worker has both encrypt and unique_for options, Sidekiq will raise an error.