Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Very poor write performance starting from CU6 #355
We find out that write performance on is very poor on CU6 (and all versions after that) if you compare it to CU5.
Steps to reproduce
Correctly working version (CU5)
Start SQL server using command:
Create test table:
Run insert test:
it should finish on couple of minutes.
Slow version (CU6)
Stop and remove test container:
Re-create test container but now use docker image
You will notice that test will run much longer and you can also see with iotop that sqlservr is writing data much slower.
We also tried to disable these settings https://support.microsoft.com/en-us/help/4131496/enable-forced-flush-mechanism-in-sql-server-2017-on-linux but they didn't help (but I'm also not sure if container version reads settings from /var/opt/mssql/mssql.conf file).
@olljanat - Thanks for reporting this issue. I was able to confirm the performance degradation between CU5 and CU6 on my macbook. It does seem to be resolved in 2017-latest (CU 10) though. Can you please confirm by running your test against :2017-latest in your env?
Here are my results:
:2017-latest (CU10) - Try 1
:2017-latest (CU10) - Try 2
Ok. CU10 is not listed on info text on https://hub.docker.com/r/microsoft/mssql-server-linux/ so I missed it totally and only tested until CU9.
I will try CU10 on tomorrow.
@twright-msft I tested :2017-CU10 on two different environments and looks that issue still exists on that.
Are you sure that you don't have some older CU version stored with :2017-latest tag to your machine?
Result of SELECT @@Version query on my env is:
Hello @olijanat. The poor write performance due to a change we made in CU6 to force a flush to disk to address a data loss exposure with the FUA issues on Linux. We have a support document about this at https://support.microsoft.com/en-us/help/4131496/enable-forced-flush-mechanism-in-sql-server-2017-on-linux.
You can disable this behavior if you believe that your storage subsystem can guarantee durable writes across a power loss. You can use trace flag 3979 will disable the forced flush mechanism and revert to pre-CU6 behavior.
@jamiere-msft ok, that makes sense. I think that it would be best if these settings would be settable using environment variables and listed on here: https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-configure-environment-variables?view=sql-server-2017
or is there some reason that these need to be handled different way?
@olljanat We are still evaluating how to handle this in the long-term. We are working with several of the Linux distro owners to come up with a way to ensure data durability and avoid the cache flush overhead. If this can be accomplished then we can remove the need for a trace flag completely and avoid adding any other environment variables. I will definitely take your comments back as we continue to work on this.