-
Notifications
You must be signed in to change notification settings - Fork 64
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add support for Bulk Upserting records
🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀 It's alive! It's alive! Adds the ability to insert large numbers of records all at once. (don't worry, will rebase once I'm sane again) 🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀🍀 Co-authored-by: Alex Piechowski <alex@piechowski.io>
- Loading branch information
1 parent
36e2f87
commit 278b28b
Showing
7 changed files
with
174 additions
and
4 deletions.
There are no files selected for viewing
9 changes: 9 additions & 0 deletions
9
db/migrations/20220113043033_add_unique_constraint_to_users.cr
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
class AddUniqueConstraintToUsers::V20220113043033 < Avram::Migrator::Migration::V1 | ||
def migrate | ||
create_index :users, [:name, :nickname], unique: true | ||
end | ||
|
||
def rollback | ||
drop_index :users, [:name, :nickname] | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
require "../spec_helper" | ||
|
||
describe Avram::BulkUpsert do | ||
describe "bulk upserts" do | ||
# Ideally, compiler can catch this / this should be impossible.. | ||
context "when collections mismatch" do | ||
end | ||
|
||
context "when mixed new records and updated records" do | ||
# Insert spec example. | ||
it "inserts with a hash of String" do | ||
# params = {:first_name => "Paul", :last_name => "Smith"} | ||
# insert = Avram::Insert.new(table: :users, params: params) | ||
# insert.statement.should eq "insert into users(first_name, last_name) values($1, $2) returning *" | ||
# insert.args.should eq ["Paul", "Smith"] | ||
end | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
class Avram::BulkUpsert(T) | ||
def initialize(@records : Array(T), @column_names : Array(Symbol)) | ||
@records = set_timestamps(records) | ||
end | ||
|
||
def statement | ||
<<-SQL | ||
INSERT INTO #{table}(#{fields}) | ||
(select * from unnest(#{value_placeholders})) | ||
ON CONFLICT (#{conflicts}) DO UPDATE SET #{updates} | ||
RETURNING #{returning} | ||
SQL | ||
end | ||
|
||
def args | ||
@records.map do |record| | ||
record.changed_attributes.map(&.value) | ||
end.transpose | ||
end | ||
|
||
private def conflicts | ||
@column_names.join(", ") | ||
end | ||
|
||
private def set_timestamps(collection) | ||
collection.map do |record| | ||
record.created_at.value ||= Time.utc if record.responds_to?(:created_at) | ||
record.updated_at.value = Time.utc if record.responds_to?(:updated_at) | ||
record | ||
end | ||
end | ||
|
||
private def table | ||
@records.first.table_name | ||
end | ||
|
||
private def updates | ||
update_keys = changed_attributes.flat_map(&.name) | ||
(update_keys - [:created_at]).map do |column| | ||
"#{column}=EXCLUDED.#{column}" | ||
end.join(", ") | ||
end | ||
|
||
private def returning | ||
T.column_names.join(", ") | ||
end | ||
|
||
private def changed_attributes | ||
@records.first.changed_attributes | ||
end | ||
|
||
private def fields | ||
changed_attributes.map do |key| | ||
<<-TEXT | ||
"#{key.name.to_s}" | ||
TEXT | ||
end.join(", ") | ||
end | ||
|
||
private def column_types | ||
T.database_table_info.not_nil!.columns.map do |column_info| | ||
[ | ||
column_info.column_name, | ||
column_info.data_type, | ||
] | ||
end.to_h | ||
end | ||
|
||
private def cast(column) | ||
"#{column_types[column.name.to_s]}[]" | ||
end | ||
|
||
private def cast(column : Avram::Attribute(Time)) | ||
"timestamptz[]" | ||
end | ||
|
||
private def value_placeholders | ||
changed_attributes.map_with_index(1) do |k, index| | ||
"$#{index}::#{cast(k)}" | ||
end.join(", ") | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters