Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Slow performance INSERT'ing large JSONB column in YCQL #2110
[UPDATE: This doesn't appear to be a JSONB specific issue, but rather an issue related to large CQL statements and the time to prepare/compile the statement. Using bind variables and reusing the prepared statements seems to address the performance problem. See later comment in this issue with a sample code snippet for optimal way to rewrite the program.]
Running on a RF=1 Docker created instance on a MacBook Pro (w/ SSD) and
I ran a 'sudo perf top' when this workload was running, and it showed these two functions on the top:
Looks like the compilation (re-prepare of statements) with large JSONB literals embedded in the CQL statement text is inefficient, and we are spending a lot of time in the lexer. We'll look into if there's anything we can do to optimize that.
Meanwhile, please use prepared statements as a workaround, and you'll see noticeably better results. Here is a sample snippet:
And here's the corresponding output:
Please also note that when using yb-docker-ctl, it by default gives the yb-tserver processes only 1GB of memory. For workloads with large payload (like 10MB+), would recommend setting the memory higher as Alan mention in the forum thread.
Please also note that we support only up to 32MB sizes RPCs and data sizes. Therefore, I modified the program provided by the original filer of the issue on forum to cap the size at 32MB.