You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to measure batch write performance of ClickHouse, and without changing the code of the client library, the most accurate measurement can be achieved by wrapping ExecuteNonQuery() with a stopwatch as such:
var stopwatch = Stopwatch.StartNew();
command.ExecuteNonQuery();
stopwatch.Stop();
However, I looked into the code of the ExecuteNonQuery() method and it seems that there is some processing overhead with parameters (I think? or is the overhead extremely minor even with batches of 10k rows?). So, let's say I want to edit the library's code to get the best measurement possible with as little overhead from the library itself. Where would I add my stopwatch to measure the time taken by batch write?
This seems like a good spot to me in ClickHouseCommand.cs:
Or should I go a level deeper into SendBlocks() or Write() in the Block class?
EDIT: Also, is there a way to separate the preprocessing from the actual sending of data into two separate methods? I have been modifying the code and I keep getting different errors when I try to separate SendBlocks() into its own method that I call after ExecuteNonQuery().
Currently, I am able to initialize and preprocess all commands by running ExecuteNonQuery() on each of them with the SendBlocks() being separated to a new function called ExecuteFinal(). However, I am getting errors when calling ExecuteFinal() with the server showing Unexpected packet Data received from client. My guess is that the connection object is being altered by each of the commands belonging to it, but I am not sure how.
The text was updated successfully, but these errors were encountered:
I would like to measure batch write performance of ClickHouse, and without changing the code of the client library, the most accurate measurement can be achieved by wrapping
ExecuteNonQuery()
with a stopwatch as such:However, I looked into the code of the
ExecuteNonQuery()
method and it seems that there is some processing overhead with parameters (I think? or is the overhead extremely minor even with batches of 10k rows?). So, let's say I want to edit the library's code to get the best measurement possible with as little overhead from the library itself. Where would I add my stopwatch to measure the time taken by batch write?This seems like a good spot to me in
ClickHouseCommand.cs
:Or should I go a level deeper into
SendBlocks()
orWrite()
in the Block class?EDIT: Also, is there a way to separate the preprocessing from the actual sending of data into two separate methods? I have been modifying the code and I keep getting different errors when I try to separate
SendBlocks()
into its own method that I call afterExecuteNonQuery()
.Currently, I am able to initialize and preprocess all commands by running
ExecuteNonQuery()
on each of them with theSendBlocks()
being separated to a new function calledExecuteFinal()
. However, I am getting errors when callingExecuteFinal()
with the server showingUnexpected packet Data received from client
. My guess is that the connection object is being altered by each of the commands belonging to it, but I am not sure how.The text was updated successfully, but these errors were encountered: