Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign up[YSQL] Support txn batch size for COPY to improve performance #2855
Comments
This comment has been minimized.
This comment has been minimized.
It might be useful to expose these COPY behavior options as GUCs, so that we could say e.g. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Today for improving COPY performance, we have a non-txn-copy gflag that can be used to run copy in non transactional mode. However, this does not take effect if table requires transactions (for example, if table has an index).
We should add a copy_txn_batch_size option to improve COPY performance for such transactional tables. This will reduce the batch size for transactions instead of loading all the data in a single large transaction.