Allow for transactional changesets; aka "Unit of Work"

[robconery]

Hi everyone - thanks for your work on Ecto! As I have been working with Ecto I've realized that 80% of the time I don't write isolated commands - I create Commands (big C) that execute a given transaction. This is a natural way for me to think about functional programming (using CQRS). It occurred to me that building a changeset with multiple changes for a transaction is a natural lead into the Unit of Work pattern - one that I really like.

I've forked the repo and am happy to take a stab at this idea - but I thought I would open the issue first and then a PR to work against. I could see this working something like this:

Changeset.queue(%User{}, {email: "test@test.com", name: "Rob"})
  |> Changeset.queue(%Log{}, {entry: "New User added"})
  |> Repo.transaction

Right now Repo.transaction accepts a function - for this work work it would need to accept a %UnitOfWork or whatever it makes sense to call the thing that Changeset.queue might use.

[josevalim]

Thank you @robconery. I agree something like this would be very helpful. The issue is a bit more complex though:

1. What if I want to insert one user and then update a log entry? You used "queue", which is not clear about the repo action;
2. What if updating the log entry requires the user.id information?
3. What happens if any of the steps fail?

I would write this code today as this:

    user_changeset = User.changeset(%User{}, params)
    Repo.transaction fn ->
      case Repo.insert(user_changeset) do
        {:ok, user} ->
          log_changeset = Log.changeset(%Log{user_id: user.id}, params)
          case Repo.insert(log_changeset) do
            {:ok, log} -> {user, log}
            {:error, log_changeset} -> Repo.rollback log_changeset
          end
        {:error, user_changeset} ->
          Repo.rollback user_changeset
    end

It solves all scenarios above and, while it is not complicated, it is very verbose. I believe you have F# background? In F# I would solve this problem by using computation expressions. Something like this in F#:

    transaction {
      let! user = insert userChangeset
      let! log = insert logChangeset
      return (user, log)
    }

We don't have something like computation expressions in Elixir yet. But if we ported the code above to Elixir, it would look something like this:

    user_changeset = User.changeset(%User{}, params)
    transaction user <- Repo.insert(user_changeset),
                log_changeset = Log.changeset(%Log{user_id: user.id}, params),
                log <- Repo.insert(log_changeset),
                do: {user, log}

TL;DR: this is definitely a problem, we don't have the tools to solve it elegantly in Elixir yet.

[josevalim]

It is worth mentioning though that the computation expression approach wouldn't allow you to queue N commands dynamically. The number of operations is static as we need to pass all steps explicitly to transaction. I am not sure if this is an issue in practice though and you could always call an anonymous function from one of the steps in transaction.

[robconery]

Thanks @josevalim I agree it's a complex issue. The difficult part is parity between adapters - for instance Postgres uses MVCC and it can handle transactions like this elegantly (needing an ID back) whereas Mongo has a different model.

Anyway - my first reaction when thinking about this is using CTEs:

with tx1 as (
  insert into my_table(...) returning *;
), tx2 as (
  insert into logs(the_id, 'a lot entry')
  select id, 'This is a log entry' from tx1;
)

The trick would be running the SQL translation first, turning it into a CTE and then handing it off to Postgres. Obviously wouldn't scale well with Ecto.

I have some ideas I'm working up in a separate repo - stuff I've been doing with .NET and Node - Ill flesh those out a bit and if I come up with something interesting I'll ping again.

Thanks again :)

[josevalim]

@robconery you were right. I was wrong. I am writing a proposal. I will copy you.

[robconery]

Cheers! There's no right and wrong in the Wild West of New Programming Langs so let's make some fun happen :)