-
Notifications
You must be signed in to change notification settings - Fork 297
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MySQL on duplicate key update #674
Changes from all commits
05293e2
19b0a6d
a1c379b
b1d5bb5
6bed164
3a1e545
696ab24
48e27e0
f12f3a0
9cea669
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
{-# LANGUAGE ConstraintKinds #-} | ||
{-# LANGUAGE GADTs #-} | ||
{-# LANGUAGE OverloadedStrings #-} | ||
{-# LANGUAGE TypeFamilies #-} | ||
{-# LANGUAGE FlexibleContexts #-} | ||
|
@@ -14,6 +15,9 @@ module Database.Persist.MySQL | |
, MySQLBase.defaultSSLInfo | ||
, MySQLConf(..) | ||
, mockMigration | ||
, insertOnDuplicateKeyUpdate | ||
, insertManyOnDuplicateKeyUpdate | ||
, SomeField(..) | ||
) where | ||
|
||
import Control.Arrow | ||
|
@@ -1014,3 +1018,107 @@ mockMigration mig = do | |
result = runReaderT . runWriterT . runWriterT $ mig | ||
resp <- result sqlbackend | ||
mapM_ T.putStrLn $ map snd $ snd resp | ||
|
||
-- | MySQL specific 'upsert'. This will prevent multiple queries, when one will | ||
-- do. | ||
insertOnDuplicateKeyUpdate | ||
:: ( PersistEntityBackend record ~ BaseBackend backend | ||
, PersistEntity record | ||
, MonadIO m | ||
, PersistStore backend | ||
, backend ~ SqlBackend | ||
) | ||
=> record | ||
-> [Update record] | ||
-> SqlPersistT m () | ||
insertOnDuplicateKeyUpdate record = | ||
insertManyOnDuplicateKeyUpdate [record] [] | ||
|
||
-- | This wraps values of an Entity's 'EntityField', making them have the same | ||
-- type. This allows them to be put in lists. | ||
data SomeField record where | ||
SomeField :: EntityField record typ -> SomeField record | ||
|
||
-- | Do a bulk insert on the given records in the first parameter. In the event | ||
-- that a key conflicts with a record currently in the database, the second and | ||
-- third parameters determine what will happen. | ||
-- | ||
-- The second parameter is a list of fields to copy from the original value. | ||
-- This allows you to specify that, when a collision occurs, you'll just update | ||
-- the value in the database with the field values that you inserted. | ||
-- | ||
-- The third parameter is a list of updates to perform that are independent of | ||
-- the value that is provided. You can use this to increment a counter value. | ||
-- These updates only occur if the original record is present in the database. | ||
insertManyOnDuplicateKeyUpdate | ||
:: ( PersistEntityBackend record ~ SqlBackend | ||
, PersistEntity record | ||
, MonadIO m | ||
) | ||
=> [record] -- ^ A list of the records you want to insert, or update | ||
-> [SomeField record] -- ^ A list of the fields you want to copy over. | ||
-> [Update record] -- ^ A list of the updates to apply that aren't dependent on the record being inserted. | ||
-> SqlPersistT m () | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This function would be used like: bulkInsertOnDuplicateKeyUpdate
[ {- a big ol' list of records you want to insert/update -} ]
[ SomeField UserName, SomeField UserEmail ] -- this copies the values that are being inserted to existing records
[ UserModified =. now, UserEncounted +=. 1 ] -- this update is performed for any field that matches the inserted records |
||
insertManyOnDuplicateKeyUpdate [] _ _ = return () | ||
insertManyOnDuplicateKeyUpdate records [] [] = insertMany_ records | ||
insertManyOnDuplicateKeyUpdate records fieldValues updates = | ||
uncurry rawExecute $ mkBulkInsertQuery records fieldValues updates | ||
|
||
-- | This creates the query for 'bulkInsertOnDuplicateKeyUpdate'. It will give | ||
-- garbage results if you don't provide a list of either fields to copy or | ||
-- fields to update. | ||
mkBulkInsertQuery | ||
:: (PersistEntityBackend record ~ SqlBackend, PersistEntity record) | ||
=> [record] -- ^ A list of the records you want to insert, or update | ||
-> [SomeField record] -- ^ A list of the fields you want to copy over. | ||
-> [Update record] -- ^ A list of the updates to apply that aren't dependent on the record being inserted. | ||
-> (Text, [PersistValue]) | ||
mkBulkInsertQuery records fieldValues updates = | ||
(q, recordValues <> updsValues) | ||
where | ||
fieldDefs = map (\x -> case x of SomeField rec -> persistFieldDef rec) fieldValues | ||
updateFieldNames = map (T.pack . escapeDBName . fieldDB) fieldDefs | ||
entityDef' = entityDef records | ||
entityFieldNames = map (T.pack . escapeDBName . fieldDB) (entityFields entityDef') | ||
tableName = T.pack . escapeDBName . entityDB $ entityDef' | ||
recordValues = concatMap (map toPersistValue . toPersistFields) records | ||
recordPlaceholders = commaSeparated $ map (parenWrapped . commaSeparated . map (const "?") . toPersistFields) records | ||
fieldSets = map (\n -> T.concat [n, "=VALUES(", n, ")"]) updateFieldNames | ||
upds = map mkUpdateText updates | ||
updsValues = map (\(Update _ val _) -> toPersistValue val) updates | ||
q = T.concat | ||
[ "INSERT INTO " | ||
, tableName | ||
, " (" | ||
, commaSeparated entityFieldNames | ||
, ") " | ||
, " VALUES " | ||
, recordPlaceholders | ||
, " ON DUPLICATE KEY UPDATE " | ||
, commaSeparated (fieldSets <> upds) | ||
] | ||
|
||
-- | Vendored from @persistent@. | ||
mkUpdateText :: PersistEntity record => Update record -> Text | ||
mkUpdateText x = | ||
case updateUpdate x of | ||
Assign -> n <> "=?" | ||
Add -> T.concat [n, "=", n, "+?"] | ||
Subtract -> T.concat [n, "=", n, "-?"] | ||
Multiply -> T.concat [n, "=", n, "*?"] | ||
Divide -> T.concat [n, "=", n, "/?"] | ||
BackendSpecificUpdate up -> | ||
error . T.unpack $ "BackendSpecificUpdate " <> up <> " not supported" | ||
where | ||
n = T.pack . escapeDBName . fieldDB . updateFieldDef $ x | ||
|
||
commaSeparated :: [Text] -> Text | ||
commaSeparated = T.intercalate ", " | ||
|
||
parenWrapped :: Text -> Text | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These functions make the code much more readable, nice idea There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks! I can use them in the other instances of these if you'd like. |
||
parenWrapped t = T.concat ["(", t, ")"] | ||
|
||
-- | Gets the 'FieldDef' for an 'Update'. Vendored from @persistent@. | ||
updateFieldDef :: PersistEntity v => Update v -> FieldDef | ||
updateFieldDef (Update f _ _) = persistFieldDef f | ||
updateFieldDef BackendUpdate {} = error "updateFieldDef did not expect BackendUpdate" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The documentation for the third parameter is kind of confusing to me
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a little difficult to explain. If you pass something like
then it generates the following clause in the Update:
So the name gets copied from whatever record was being inserted and had a key collision, and the age gets incremented by 1.
Since the
Update
field requires a specific value, that means that we can't make it dependent on the record that we're updating. MySQL would support something likewhich would add 2 to the record that hit the duplicate key. There's not a good way to make the update have more complicated values, unfortunately, so this feature would be difficult to incorporate as-is.