New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When execute command seed:run
, insert all data to a table in a bulk.
#1148
Conversation
It take much time to insert data to a table one by one until now. An example is shown below: ``` $table = $this->table('users'); $table->bulk() ->insert($data) ->save(); ``` It became more than triple as fast by Bulk Insert when inserting 1,000 records. It is very effective when many data samples are required.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way you can implement this without adding the bulk flag? I'm not sure it makes for a good API
src/Phinx/Db/Table.php
Outdated
@@ -75,6 +75,11 @@ class Table | |||
protected $data = array(); | |||
|
|||
/** | |||
* @var boolean | |||
*/ | |||
protected $bulk = false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand what this flag is for
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This flag is prepared to judge whether to use the bulk insert function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not always use it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see.
There is a way I can implement this without adding the bulk flag.
Thx.
2c27f6a
to
7368363
Compare
src/Phinx/Db/Adapter/PdoAdapter.php
Outdated
$keys = array_keys($current); | ||
$sql .= "(". implode(', ', array_map(array($this, 'quoteColumnName'), $keys)) . ") VALUES"; | ||
|
||
$objTmp = (object) array('aFlat' => array()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
aFlat
Typo?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, typo !
fix it !
src/Phinx/Db/Adapter/PdoAdapter.php
Outdated
$sql .= "(". implode(', ', array_map(array($this, 'quoteColumnName'), $keys)) . ") VALUES"; | ||
|
||
$objTmp = (object) array('aFlat' => array()); | ||
array_walk_recursive($rows, create_function('&$v, $k, &$t', '$t->flat[] = $v;'), $objTmp); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why don't you use a Closure? Phinx seems to require PHP >= 5.4. And if you use a closure, $objTmp
is unnecessary, as it can inherit variables form the parent scope by reference. And $rows
would be a two-dimensional array. You need not handle it recursively. If I were you, I would use foreach
in this case:
$vals = [];
foreach ($rows as $row) {
foreach ($row as $v) {
$vals[] = $v;
}
}
Also, I think you should make sure that all rows have the same keys. I think it doesn't make a big impact on performance, but dose prevent users' mistakes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your review !
I tried to simply coded in one line.
But there was not enough consideration to Phinx.
Fix it !
Thx.
fallaten values in data simply. And typo, sorry.
set prefix adapter to a table when bulk insert.
By checking the keys in data to be inserted, and when all the keys are matched, execute bulk insert.
By checking the keys in data to be inserted, and when all the keys are matched, execute bulk insert.
… into feature/bulkinsert
I could implement Bulk Insert without adding the bulk flag 🤓 |
I think the only thing this is missing is tests showing that the method actually works |
Thank you for review ! I added to test the new method with the following method.
Is it necessary elsewhere ? |
@kenzo0107 I didn't see any tests where the method was actually used. Only mocked tests were used, so the actual implementation is not tested |
add test of method bulkinsert to each adaptor.
@lorenzo Thank you for your review. I added some tests where the method is actually used. |
Thanks @kenzo0107 ! |
@kenzo0107 Would you mind sending another pull request with a short example for the docs? |
@lorenzo Thank you 😄 I will send another pull request soon. |
When execute command
seed:run
, insert all data to a table in a bulk.It take much time to insert data to a table one by one until now.
An example is shown below:
It became more than triple as fast by Bulk Insert when inserting 1,000 records.
It is very effective when many data samples are required.
However, it is assumed that the keys of data to be set are unified.
The method
bulk()
are not available in the following example:The function of dynamically checking the key of the set data is not added,
because it will cause performance degradation.