Recommendations for data migrations

[tielur]

So I recently started using cloak in a project with existing data. I'm in the process of figuring out the best way to migrate that data to it's encrypted version. I ran into a few things and I'm looking for recommendations that might be worth documenting for other users. Here's a few things:

1. I want to encrypt data in a migration.
2. I want to use schemaless Ecto queries to future proof the migrations ( changing the model in the future would cause a schema based query to fail)

I originally went the route of #1 and just wrote a query to fetch data and then update it so that Cloak would encrypt the data. This worked ok but because it's using Schema queries if you want to update the model in the future like I am doing now, the migration will fail. This lead me down the route of schemaless queries to be used in migrations. Which everyone seems to be using anyways for this reason.

So if we go with schemaless queries, I don't think I can go the route of just updating the data and expecting Cloak to encrypt it. Here's where I need a suggestion on how to handle the encryption. Do I just use the manual encrypt methods to handle that?

Is this a common enough problem that maybe Cloak should include an example of how to handle this?

[tielur]

Another option would be to define a schema in the migration to use with a schema query?

[danielberkompas]

I agree that there should be documentation on how to migrate existing data. I think this would consist of calling the encrypt functions manually, yes.

[danielberkompas]

Guidance will be coming for this in the upcoming v0.7.0 release. Stay tuned.

[tielur]

For what it's worth I went with defining a schema in my migration that worked well. Ideally this probably shouldn't even be in a migration, probably makes more sense in a mix task. I'm looking forward to see what solution Cloak takes though. Thanks for the update!

[danielberkompas]

@tielur this is the guide that will be included in Cloak 0.7:

https://github.com/danielberkompas/cloak/blob/master/guides/how_to/encrypt_existing_data.md

[tielur]

@danielberkompas Awesome looks good. I like keeping that in a mix task as well. 👍