Support clean-slate encryption & decryption (w/ zero-downtime)

[ribose-jeffreylau]

This basically means migrating a column from unencrypted to encrypted, and vice versa.

It would be invaluable for a project to be able to safely encrypt data from existing columns that are not yet encrypted, or to be able to e.g. declassify data. This feature would seem like a natural step.

In terms of zero-downtime, it means supporting things like:

class User < ActiveRecord::Base
  attr_transcryptor :ssn,
                    old: {
                      key: proc { |user|
                        ENV['OLD_USER_SSN_ENC_KEY'] || ENV['USER_SSN_ENC_KEY']
                      },
                      mode: :per_attribute_iv,
                      algorithm: 'aes-256-gcm'
                    },
                    new: {}

  # ...
end

^ Read from the old column, but write to both old and new columns. The new column (e.g. ssn) is written _un_encrypted.

class User < ActiveRecord::Base
  attr_transcryptor :ssn,
                    new: {
                      key: proc { |user|
                        ENV['OLD_USER_SSN_ENC_KEY'] || ENV['USER_SSN_ENC_KEY']
                      },
                      mode: :per_attribute_iv,
                      algorithm: 'aes-256-gcm'
                    },
                    old: {}

  # ...
end

^ As above, but the new column (e.g. encrypted_ssn) is written _en_crypted.

And from a Migration's perspective, it means something like:

        # Encrypt column
        re_encrypt_column(
          :my_table,
          :column_1,
          {},
          { key: '2asd2asd2asd2asd2asd2asd2asd2asd' }
        )

        # Decrypt column
        re_encrypt_column(
          :my_table,
          :column_1,
          { key: '2asd2asd2asd2asd2asd2asd2asd2asd' },
          {}
        )

[ronaldtse]

Exactly what we need 👍

[ribose-jeffreylau]

Hi @DmitryDrobotov , could you also look into implementing this for us? Thanks! :)

[ribose-jeffreylau]

@DmitryDrobotov Welcome back! 🎉 :)

[ribose-jeffreylau]

Hi @nattfodd , are we able to build on #35 to support going from non-encrypted to encrypted and vice versa? IIUC, we would still need separate database migrations for creating / removing the ***_salt, ***_iv and ***_version columns? Thanks!

[nattfodd]

@ribose-jeffreylau

are we able to build on #35 to support going from non-encrypted to encrypted and vice versa?

It is needed to be tested out, but I believe we're not (yet). I tested only the case with different encryption options.

IIUC, we would still need separate database migrations for creating / removing the ***_salt, ***_iv and ***_version columns?

yeap

[ribose-jeffreylau]

Thanks @nattfodd for the clarifications! 👍

[ronaldtse]

(Comment ported from #35 )

Regarding unencrypted to encrypted, I suspect the following naming pairs make better sense?

Data column	Version column	Nature
ssn	ssn_version	Source
encrypted_ssn	encrypted_ssn_version	Destination

So the database will:

1. Start with an :ssn column
2. Add a database migration to add :encrypted_ssn, :encrypted_ssn_version columns (and other necessary attr_encrypted columns) (and run it)
3. Add a data migration to migrate :ssn data to :encrypted_ssn

Perhaps something like this is reasonable?

[ribose-jeffreylau]

Hi, just curious about the status of this. Is this being worked on? What else is needed?

[nattfodd]

HI, @ribose-jeffreylau It's done basically

[ribose-jeffreylau]

Thanks @nattfodd ! Does that mean the following bit would serve to migrate from unencrypted to encrypted? (after having manually done steps 1 and 2 from #24 (comment)) :

  version 1
  version(2) do
    decode_with_previous_settings(migration,
      key:       '67c3800d1572d9d964a6ff3bd821ed02',
      algorithm: 'aes-256-gcm'
    )
  end

And the following would go from encrypted to unencrypted?

  version(2) do
    decode_with_previous_settings(migration,
      key:       '67c3800d1572d9d964a6ff3bd821ed02',
      algorithm: 'aes-256-gcm'
    )
  end
  version 3