New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bulk inserts #179

Closed
jschoch opened this Issue Feb 24, 2014 · 9 comments

Comments

Projects
None yet
7 participants
@jschoch

jschoch commented Feb 24, 2014

please add bulk inserts

if possible i'd like to bulk load relationships as quickly as possible though i'm not sure how this is best done in posgresql

@ericmj

This comment has been minimized.

Member

ericmj commented Feb 24, 2014

This requires extending the postgresql driver with support for all steps in extended queries. I opened an issue: elixir-ecto/postgrex#7.

It also requires that ecto use prepared statements for queries. Issue open here: #180.

@ericmj ericmj added the Feature label Mar 10, 2014

@ericmj ericmj closed this Aug 27, 2014

@ericmj

This comment has been minimized.

Member

ericmj commented Aug 27, 2014

Closing until we have support in postgrex

@jclem

This comment has been minimized.

jclem commented Jul 6, 2015

This thread was a little hard to follow, but it seems this is in Postgrex, now. Is it possible to do bulk inserts in Ecto, yet?

@josevalim

This comment has been minimized.

Member

josevalim commented Jul 6, 2015

No, it is not possible. Unless someone tackles this from beginning to finish, we have no plans to include it in 1.0.

@loganhenson

This comment has been minimized.

loganhenson commented May 27, 2016

For those who end up here and want an update:

via @michalmuskala

There's Repo.update_all, ecto 2.0 also has Repo.insert_all

https://hexdocs.pm/ecto/Ecto.Repo.html#c:update_all/3

@edevil

This comment has been minimized.

Contributor

edevil commented Aug 10, 2016

Repo.insert_all doesn't work with a changeset list. Will this be supported in the future?

@jclem

This comment has been minimized.

jclem commented Aug 10, 2016

Changesets can have associations on them, which doesn't make sense to me in the context of a bulk insert. I guess Ecto could collate the associations somehow and perform multiple bulk inserts, but seems pretty difficult.

@michalmuskala

This comment has been minimized.

Member

michalmuskala commented Aug 10, 2016

It's not planned to add changeset support to Repo.insert_all. Repo.insert_all issues a single database INSERT statement, which is not compatible with many features of the changesets - most notably prepare_changes and associations. Repo.insert_all is designed (like all *_all functions) as a low level operation with very predictable and clear translation to the database operations.

@edevil

This comment has been minimized.

Contributor

edevil commented Aug 16, 2016

FTR, this is what I'm doing now:

    multi =
      changesets
      |> Enum.reduce(Multi.new, fn(cset, multi) -> Multi.insert(multi, Ecto.UUID.generate, cset) end)

    {:ok, _} = Repo.transaction(multi)

Of course, it's still doing multiple inserts. It would be nice to have a straightforward way of taking a bunch of changesets that only touch one table and bulk-inserting them. Maybe validating the changesets and converting them to a list of changes that can be passed to insert_all?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment