New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LOAD DATA LOCAL INFILE implemetation #158
Conversation
Thanks! I will merge this after fix failed tests in master. And I prefer to cherry-pick commits manually to avoid duplications from your fork. You can help me with this if you have some time. |
void MysqlConnection::RestoreLocalInfileHandlers(local_infile_data * infile_data, | ||
MYSQL * conn) { | ||
if (infile_data) { | ||
mysql_set_local_infile_default(conn); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Мы не должны вызвать mysql_thread_init();
перед этой строкой?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Хм, всё равно это восстановление и если дальше будет использоваться стандартный обработчик - в его инициализации функция всё равно вызовется. Но всё равно странный порядок вызовов:
mysql_set_local_infile_default(conn);
...
mysql_thread_init();
mysql_set_local_infile_default(conn);
Так что пусть будет.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
В случае стандартного load infile (без буфера), последовательность становиться
mysql_thread_init();
mysql_set_local_infile_default(conn);
mysql_thread_end(); //чтобы не было memory leak если в libuv в пуле потоков какой то закроется
в случае наличия буфера последовательность
mysql_set_local_infile_handler(...)
// не зависит от mysql_thread_init
mysql_set_local_infile_default(conn);
// в принципе не очень хорошо, так как стандартная реализация зависит от
// mysql_thread_init(), но не очень и плохо, так как в следующем запросе,
// в случае надобности, будет вызван mysql_thread_init
As I understand, you are busy with some bugs with Travis, but if there will be any problems with this pull request, please, write. I would like to help, as this feature is critical for me (it makes massive inserts into database 3-5 times faster that simple inserts) |
Done!
Can you rebase your branch onto upstream master? |
add optional argument to query/querySync method, with buffer, that is used instead of file in load data infile local operation TODO: test syncronous version of all this test if normal load data infile local works remove set option MYSQL_OPT_LOCAL_INFILE from InitLocalInfileCallbacks
@@ -1081,8 +1079,15 @@ void MysqlConnection::EIO_Query(uv_work_t *req) { | |||
query_req->connection_closed = false; | |||
|
|||
MYSQLCONN_DISABLE_MQ; | |||
|
|||
pthread_mutex_lock(&conn->query_lock); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now this is second call in this function that causes dead lock.
Good work, Anton! |
features from mysqlclient library
…ad infile handlers) work in syncronous and asyncronous mode
Thanks, don't forget to update code, because I've just fixed bug (my, incorrect rebase) with locks (double lock in EIO_Query); |
Now passes :) |
LOAD DATA LOCAL INFILE implemetation
(feature similar to postgresql's COPY FROM STDIN). Sometimes it's more convenient that using temporary file or pipe.
This feature requires update in query and querySync interface (without breaking compatibility)
Examples: