New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: insert batch rewrite. #580

Closed
wants to merge 22 commits into
base: master
from

Conversation

Projects
None yet
5 participants
@Chrriis
Contributor

Chrriis commented Jun 7, 2016

I added a new test which does not pass because batch rewrite assumes the values to add have a number of parameters equal to the number of dynamic parameters. This omits the existence of fixed parameters.
So a statement like:
INSERT INTO testbatch VALUES (?,1)
becomes:
INSERT INTO testbatch VALUES ($1,1),($2),($3)
I guess it should replicate the whole block, which should become:
INSERT INTO testbatch VALUES ($1,1),($2,1),($3,1)

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 7, 2016

Looks good.
Would you implement the fix as well?

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 7, 2016

I might give it a try but I am not sure yet what the best approach is for a fix.

I have the feeling that in BatchedQueryDecorator:246, we should find in "s" whatever is after the "VALUES" token and replicate that as many times as needed. If I am not mistaken, we cannot have anything fancy after the VALUES token, because we only get a BatchedQueryDecorator if it is simple enough to be rewritten.

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 7, 2016

we cannot have anything fancy after the VALUES token

Technically speaking, there's ?, ? -> $1, $2 kind of translation somewhere, so it might be a bit tricky to replace ?s properly.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 7, 2016

I added a fix. From what I understood, only a single set of parentheses is allowed for the batch to be rewritten. So I can safely consider the last set of parentheses to be the block of values. I then rewrite that block, renumbering the parameters using the expected index.

@codecov-io

This comment has been minimized.

codecov-io commented Jun 7, 2016

Current coverage is 58.08%

Merging #580 into master will decrease coverage by 0.03%

  1. 2 files in ...g/postgresql/core/v3 were modified. more
    • Misses +4
    • Hits -4
  2. 1 files in .../org/postgresql/core were modified. more
    • Misses +1
    • Hits -1
  3. 1 files in .../java/org/postgresql were modified. more
    • Misses +1
    • Hits -1
  4. File .../TimestampUtils.java (not in diff) was modified. more
    • Misses -1
    • Hits +1
@@             master       #580   diff @@
==========================================
  Files           147        147          
  Lines         15507      15517    +10   
  Methods           0          0          
  Messages          0          0          
  Branches       3064       3073     +9   
==========================================
  Hits           9013       9013          
- Misses         5280       5282     +2   
- Partials       1214       1222     +8   

Powered by Codecov. Last updated by 1eb4085...2165fe8

for (int j = startIndex; j < endIndex; j++) {
char ch = s.charAt(j);
switch (ch) {
case '$':

This comment has been minimized.

@vlsi

vlsi Jun 7, 2016

Member

Will it blend for values (?, 'dollar here: $', ?) kind of statement?

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 7, 2016

Ah, it seems to require more involved parsing so values (?, 'literals with ?, (, $, and )', ?) works.
We have org.postgresql.core.Parser that handles parsing to some degree.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 7, 2016

In fact you are right. These do not pass:
VALUES (?, 'literals with ?, (, $1234, and )', ?)
VALUES ($1,'1, (, $1234, a''n?d )') /xxxx)/

I can improve but this is a slippery slope into writing a complete parser.

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 7, 2016

I can improve but this is a slippery slope into writing a complete parser.

Do you realize pgjdbc has to follow that route for each and every PreparedStatement?
It has to somehow parse the SQL and replace ? bind placeholders with $n style ones (with account for comments and literals).
Frankly speaking, that was a bit surprising for me when I realized that for a first time.

Having said that, I see the following options:

  1. Add test (and more extreme variations) with @Ignore annotation, so somebody could pick it up later
  2. Teach batch rewriter to rewrite only those statements that are simple values (?,?,?) (that is just questionmarks commas and spaces after values)
  3. Reuse postgresql.core.Parser and rewrite with account for comments/literals.
@polobo

This comment has been minimized.

polobo commented Jun 7, 2016

On Tue, Jun 7, 2016 at 12:57 PM, Vladimir Sitnikov <notifications@github.com

wrote:

I can improve but this is a slippery slope into writing a complete parser.

Do you realize pgjdbc has to follow that route for each and every
PreparedStatement?
It has to somehow parse the SQL and replace ? bind placeholders with $n
style ones (with account for comments and literals).
Frankly speaking, that was a bit surprising for me when I realized that
for a first time.

Having said that, I see the following options:

  1. Add test (and more extreme variations) with @ignore annotation, so
    somebody could pick it up later
  2. Teach batch rewriter to rewrite only those statements that are simple values
    (?,?,?) (that is just questionmarks commas and spaces after values)
  3. Reuse postgresql.core.Parser and rewrite with account for
    comments/literals.

FTR I'd be concerned about unintended consequences while doing #2 but from
a usability perspective I would not complain if I had to operate within
that restriction. Pulling constants out and placing them in the client
Java code is trivial and you already have one parameter you are dealing
with so the overhead is sunk.

Also, these should not be considered mutually exclusive.​

David J.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 7, 2016

Do you realize pgjdbc has to follow that route for each and every PreparedStatement?

Yes I do :)
What a pity that the parsing it did has to be done again in case of the batch rewriting.

About 1), of course, this is something I should do.

About 2), where I work we use comments in queries, because we do query rewrite to target alternate views than the indicated tables under certain conditions. So ignoring the case of comments would greatly reduce the usefulness of that feature for us.

I am trying 3) but I see that certain parsing like parseSingleQuotes require parameters which I am not sure where to find if I were to replicate such parsing in the context of the batch rewriter.
I am also tempted to keep track of certain key positions from the initial parsing, but then I am not sure where this data should go to be picked up again at rewriting time.

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 7, 2016

I am also tempted to keep track of certain key positions from the initial parsing

Frankly speaking, "batch rewriter" is in my "to review" list. I'm not familiar with that code.

I am also tempted to keep track of certain key positions from the initial parsing,

So am I, however I'm not sure if that's feasible. On the other hand, it looks like int valuesBraceOpenPosition, valuesBraceClosePosition; int[] questionMarkPositions should be enough to cerate proper multivalues.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 7, 2016

On the other hand, it looks like int valuesBraceOpenPosition, valuesBraceClosePosition; int[] questionMarkPositions should be enough to cerate proper multivalues.

This is exactly what I am trying to do.
In BatchedQueryDecorator, I have access to "getNativeQuery().bindPositions", which seem to be the equivalent of "questionMarkPositions".
I need "valuesBraceOpenPosition, valuesBraceClosePosition", which I might be able to extract/compute in QueryExecutorImpl to pass to the BatchedQueryDecorator.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 7, 2016

I just pushed some code to fix various issues. I had to go a bit further than I had expected, so please let me know if I did things in unexpected ways.
So now, during parsing, we don't set a flag to indicate batch rewrite anymore, but we set the start and end positions of the values parameters block (start/end parentheses). Because parsing already stored the positions of the original parameters to bind and we know the start and end of the block, we know exactly which parts of the query to replicate (the chunks around the parameters within the values block).
I also improved parsing because the keyword detection logic was creating too many false positive (for example "AVALUES" entered the "VALUES" logic).

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 7, 2016

Note: I don't know what to do with the test "DeepBatchedInsertStatementTest.testNativeSqlSizeCalculation" because size calculation needs to know the block information now.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 8, 2016

On my real world setup, the new code fails, but not because of parsing. The general idea is that every batch adds a new block of parameters, but there is no place where it says it should stop and start a new batch. The first of my real world cases has:

  • 22 dynamic parameters, 1 fixed parameter.
  • 16'450 batches.
  • Which means 361'900 parameters after rewrite.
  • The resulting statement is 3'212'254 characters long.

And I get the error: SQL State 08006
"java.io.IOException: Tried to send an out-of-range integer as a 2-byte value: 361900"

I guess Postgres has an upper limit of number of parameters around 65536 (if unsigned). In any case, I think Postgres performance is best with about 100 blocks of values (not 16450!) so we should aggregate with max of 100 (or any magical number if it should depend on number of parameters). In my original external logic (that is before I tried to fix the driver feature), I used a formula like: "Math.min((2000 - 1) / parameterCount, 100)".

I am not sure how to start a new batch every 100 parameters, and I am not even sure it is possible. If it is not possible to divide a query into multiple queries, then the whole feature can be thrown away. Any ideas?

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 8, 2016

I am not sure how to start a new batch every 100 parameters, and I am not even sure it is possible

Well, we should split on values boundaries. That should work.

I think Postgres performance is best with about 100 blocks of values

According to #491 (comment), the gain from 16 -> 128 values is not that high, so having something like "Math.min(100, 65535/num_binds) values max" makes sense.

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 8, 2016

I guess Postgres has an upper limit of number of parameters around 65536

The number of parameters is indeed just int16. Not sure if that's signed or not from backend's point of view:

https://www.postgresql.org/docs/9.5/static/protocol-message-formats.html
Int16
The number of parameter format codes that follow (denoted C below). This can be zero to indicate that there are no parameters or that the parameters all use the default format (text); or one, in which case the specified format code is applied to all parameters; or it can equal the actual number of parameters.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 8, 2016

I am not sure how to start a new batch every 100 parameters

I actually meant every 100 blocks of parameters of course.
What I meant was that I am not sure if a new batch query can be created every 100 blocks and added to the list of queries to be processed. I will have to investigate, though I am not that familiar with the code yet.

The number of parameters is indeed just int16. Not sure if that's signed or not from backend's point of view.

I would like to limit to some reasonable number which does not result in huge statements. This number should be way below those technical limit to not have to ask whether it is signed or not :)

* @param offset position of query to start checking
* @return boolean indicates presence of word
*/
public static boolean parseDeleteKeyword(final char[] query, int offset) {

This comment has been minimized.

@whitingjr

whitingjr Jun 8, 2016

Contributor

Please don't substitute the code to coerce the char. Dave indicated this technique is necessary.
#491 (comment)

@whitingjr

This comment has been minimized.

Contributor

whitingjr commented Jun 8, 2016

@Chrriis thank you.

@whitingjr

This comment has been minimized.

Contributor

whitingjr commented Jun 8, 2016

@Chrriis I've been thinking about how to build in the splitting up of an individual statement. Avoiding too many values excepitons. At the moment the driver has the concept of a multi statement. org.postgresql.jdbc.PgStatement.batchStatements at the moment will only contain a single statement. This allows the ability to have more than one.
It should be possible to detect too many entries and build another statement. The QueryExecutorImpl will need updating to issue the extra statements to the back end.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 8, 2016

My last commit is a version that works in our production environment. My real world test case went down from 54 seconds to 28 seconds! This is better than my external implementation with which I got down to 32 seconds.

That commit handles batches of plain statements, which was not handled by the original implementation and broke with my test case (yes, it has those too). That test case is a whole sequence of select/update/insert statements (260 statements in total). It contains 18 batched insert statements, some with 23, 248, 1412, 16450, 78798 batches. Queries also have comments, static parameters, ...
If I am not mistaken, the max number of batches is actually a signed int16. But I decided to find the appropriate number of batches using my formula ("Math.min((2000 - 1) / paramCount, 100)") to avoid sql insanely long.

The final test for me would be to integrate a snapshot build containing those changes and see how our nightly integration tests work, but I have to say I am much more confident they would pass now. Please let me know if you intend to integrate these changes and make such snapshot that I could try.

@Chrriis Chrriis changed the title from test: mix dynamic and fixed parameters in batch rewrite. to fix: insert batch rewrite. Jun 9, 2016

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 9, 2016

I improved the test case, which I originally posted on the mailing list, to serve as a benchmarking tool. Here it is:
MultiInsertTest.zip

The results on my machine are (1 million rows, 11 columns, 10 parameters) :
Execution WITHOUT rewrite: 1min 14s 909ms
Execution WITH rewrite: 29s 349ms

I would say, 2.5x boost is impressive! Feel free to play with it and let me know what you find/think.

Note that I had to upload as a zip, but it only is a single standalone test that only requires Postgres driver JAR in the classpath as well as tweaking a few constants at the top of the file.

@vlsi

This comment has been minimized.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 9, 2016

@vlsi Thanks, I did not know. That being said, this does not seem to fit with what I was doing: I was showing the gains, not just measuring the current state of things. Correct me if I am wrong, but the test you pointed me to does not measure gains between 2 approaches.

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 9, 2016

Correct me if I am wrong, but the test you pointed me to does not measure gains between 2 approaches

To measure "before vs after" one just needs to run the same benchmark against different pgjdbc versions.

I was showing the gains

The gains might depend on "the number of columns", "number of rows", etc, etc. It turns out JMH allows walking the parameter space, so we can check if there is a sweet point of "100 binds" or "100 rows" or whatever.

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 9, 2016

@vlsi I think we agree but we were not on the same train of thoughts :)
I was merely providing the order of magnitude of gains, what to expect by having this feature active.
As a follow up on my findings, for 10 million rows I have more than 3 times gain (6 minutes instead of 20).

Chrriis and others added some commits Jun 9, 2016

WIP: fixup batch rewriter
* fix "values(..." case (no space after values)
* support rewrite even in autocommit=true mode
* Test insert rewriter under BatchFailureTest
* fix values((?),((?)),?); case
* fix "no more than 32768 binds" issue
* assertTrue(Arrays.equals) -> assertEquals(Arrays.toString) for easier readability
@vlsi

This comment has been minimized.

Member

vlsi commented Jun 20, 2016

Here's the fix to "pre-describe the first statement in a batch" vlsi@1d60fcf#diff-e517b352b59b7d012c0a772c7e84d3eeR965

There is a room for improvement (e.g. being able to pre-describe in the middle of the batch), however we can implement that later.

@Chrriis Chrriis force-pushed the Chrriis:ReWrittenBatch branch from 6a1b61f to 1d60fcf Jun 20, 2016

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 20, 2016

@vlsi I tested the heavy database copy test I mentioned earlier: fast, no freeze and result is as expected! So I reset my branch to your branch and forced push. I also modified the prepared statement test so that the regular batch is also tested (which passes now with your modifications).

@Chrriis

This comment has been minimized.

Contributor

Chrriis commented Jun 20, 2016

So you know, I tried a different test case and your pre-describe modifications also boost performance: I dropped from 28 seconds to 24 seconds. Great!

@Override
public String toString() {
return getNativeSql();

This comment has been minimized.

@vlsi

vlsi Jun 21, 2016

Member

BatchedQuery.toString should return not just SQL text, but the bind values as well.
It might be useful for debugging purposes (see how SimpleQuery returns SQL+binds)

This comment has been minimized.

@Chrriis

Chrriis Jun 21, 2016

Contributor

Hm, that is interesting. It is actually an aspect I hate about Postgres, for several reasons. My view is that the default should be the parameterized SQL, another method to get the parameters, and eventually another one with SQL + bind. I will come back on that another time, maybe with a pull request, who knows.
To come back to the current situation, should it show the latest bindings only?

This comment has been minimized.

@Chrriis

Chrriis Jun 21, 2016

Contributor

Oops, I got confused between statement and query String representation.
Still, looking at the code, I am not sure it shows SQL + binds. It seems to show "?" instead of "$1" though. Could we clarify exactly how it should be? Feel free to adjust the code if you want :)

This comment has been minimized.

@vlsi

vlsi Jun 21, 2016

Member

To come back to the current situation, should it show the latest bindings only?

Ah, we can end up with the same query object and different bind parameters in the same series of batch inserts.

Let's decide on that later then.

The idea with toSting is external profilers that instrument pgjdbc code to measure time and report sql/binds should have some sensible way of accessing query and binds without too much hard-coding for a specific pgjdbc version.
I do support an in-house profiler, and it is not a nice thing to reimplement "support of pgjdbc" as it gets upgraded. That is why I tried to rely on simple toString.

It might be time to introduce some proper API (that will cover sql, binds, batches, etc)

This comment has been minimized.

@vlsi

vlsi Jun 21, 2016

Member

Still, looking at the code, I am not sure it shows SQL + binds. It seems to show "?" instead of "$1"

My bad. my local time is ~1am, and I should probably just have some sleep.

I use Query.toString for sql, and ParameterList for parameters. Current BatchQuery implementation should be just fine.

This comment has been minimized.

@Chrriis

Chrriis Jun 21, 2016

Contributor

It might be time to introduce some proper API (that will cover sql, binds, batches, etc)

I like to hear that! I developped a low level (from our stack point of view) logger/debugger and it worked fine with other RDBMS because I could easily extract the SQL, the current set of parameters when adding the batch and a few other things. With Postgres we are blind. I added several layers of reflection but it is ugly and breaks at each upgrade. Worse: I cannot decode the parameters to see a float8 value for example. Last time I looked, the closest was logic coded within ResultSet...

This comment has been minimized.

@vlsi

vlsi Jun 21, 2016

Member

With Postgres we are blind

Feel free to raise an issue on that

This comment has been minimized.

@Chrriis

Chrriis Jun 22, 2016

Contributor

Feel free to raise an issue on that

Done: #590
Maybe the discussion from that issue will lead to a clear decision as to how statements/queries should behave. Then we can improve what we currently have.

int offset = 0;
// In case of rewrite, we split into various queries
// It would be weird to have "no info" for some and 1 for another.
boolean isMultiValueBatchRewrite = queries[0].getBatchSize() > 1;

This comment has been minimized.

@vlsi

vlsi Jun 21, 2016

Member

@Chrriis , are there any specific problems why you made isMultiValueBatchRewrite global instead of per-query?
Technically speaking, at this line, hasRewrites==true, so it is strange that you recalculate isMultiValueBatchRewrite.

I think of reverting to per-query if (batchSize>1)

This comment has been minimized.

@Chrriis

Chrriis Jun 21, 2016

Contributor

If I remember correctly, it was not a technical issue but rather nonsense from a user point of view.
1 batch = 1 row. Fine.
2 batches = 2 no info.
3 batches = 2 no info, 1 row.
...
6 batches = 6 no info.
7 batches = 6 no info, 1 row.

Good luck with explaining users that behavior :)

But using hasRewrites is probably better than isMultiValueBatchRewrite.

vlsi added a commit that referenced this pull request Jun 21, 2016

fix: improve insert values batch rewrite
Now it supports VALUES (?,1), and splits very long batch into series of smaller ones
to keep overall number of bind variables <32768 (that is PG's limit in v3 protocol)

Trigger property: reWriteBatchedInserts=true. Default value is still "false".

closes #580
closes #584

vlsi added a commit that referenced this pull request Jun 21, 2016

fix: improve insert values batch rewrite
Now it supports VALUES (?,1), and splits very long batch into series of smaller ones
to keep overall number of bind variables <32768 (that is PG's limit in v3 protocol)

Trigger property: reWriteBatchedInserts=true. Default value is still "false".

closes #580
closes #584

vlsi added a commit that referenced this pull request Jun 22, 2016

fix: improve insert values batch rewrite
Now it supports VALUES (?,1), and splits very long batch into series of smaller ones
to keep overall number of bind variables <32768 (that is PG's limit in v3 protocol)

Trigger property: reWriteBatchedInserts=true. Default value is still "false".

closes #580
closes #584

vlsi added a commit that referenced this pull request Jun 22, 2016

fix: improve insert values batch rewrite
Now it supports VALUES (?,1), and splits very long batch into series of smaller ones
to keep overall number of bind variables <32768 (that is PG's limit in v3 protocol)

Trigger property: reWriteBatchedInserts=true. Default value is still "false".

closes #580
closes #584
@vlsi

This comment has been minimized.

Member

vlsi commented Jun 22, 2016

Just in case, here are some more measurements:

Medium batch sizes:

Benchmark                           (p1nrows)  (p2multi)  Mode  Cnt    Score    Error  Units
InsertBatch.insertBatchWithRewrite       1024          1  avgt   10    2,672 ±  0,185  ms/op
InsertBatch.insertBatchWithRewrite       1024          4  avgt   10    3,204 ±  0,911  ms/op
InsertBatch.insertBatchWithRewrite       1024          8  avgt   10    2,674 ±  0,236  ms/op
InsertBatch.insertBatchWithRewrite       1024         16  avgt   10    2,711 ±  0,281  ms/op
InsertBatch.insertBatchWithRewrite       1024        128  avgt   10    2,603 ±  0,185  ms/op

Big batch sizes: response time is comparable (yet we might consider batching more than 128 values), and "rely on rewriter" approach allocates more heap memory (that might be due ot SimpleParameterList objects being created for each addBatch)

Benchmark                                  (p1nrows)  (p2multi)  Mode  Cnt         Score      Error   Units
insertBatchWithRewrite                        102400          1  avgt    5       246,102 ±   64,468   ms/op
insertBatchWithRewrite:·gc.alloc.rate.norm    102400          1  avgt    5  44308466,080 ±  349,433    B/op
insertBatch                                   102400       1024  avgt    5       223,879 ±   47,590   ms/op
insertBatch:·gc.alloc.rate.norm               102400       1024  avgt    5  26063068,480 ±  279,578    B/op
insertBatch                                   102400          1  avgt    5       966,552 ±  161,170   ms/op
insertBatch:·gc.alloc.rate.norm               102400          1  avgt    5  58493524,800 ± 1625,625    B/op

Reference:

Benchmark                           (p1nrows)  (p2multi)  Mode  Cnt    Score    Error  Units
InsertBatch.insertBatch                    16        128  avgt   10    0,190 ±  0,020  ms/op
InsertBatch.insertBatch                   128        128  avgt   10    0,452 ±  0,049  ms/op
InsertBatch.insertBatch                  1024          1  avgt   10    9,530 ±  1,896  ms/op
InsertBatch.insertBatch                  1024          4  avgt   10    4,368 ±  0,359  ms/op
InsertBatch.insertBatch                  1024          8  avgt   10    3,283 ±  0,233  ms/op
InsertBatch.insertBatch                  1024         16  avgt   10    2,946 ±  0,477  ms/op
InsertBatch.insertBatch                  1024        128  avgt   10    2,519 ±  0,341  ms/op
InsertBatch.insertBatchWithRewrite         16        128  avgt   10    0,191 ±  0,026  ms/op
InsertBatch.insertBatchWithRewrite        128        128  avgt   10    0,442 ±  0,038  ms/op
InsertBatch.insertCopy                     16        128  avgt   10    0,206 ±  0,016  ms/op
InsertBatch.insertCopy                    128        128  avgt   10    0,330 ±  0,082  ms/op
InsertBatch.insertCopy                   1024          1  avgt   10  209,068 ± 15,828  ms/op
InsertBatch.insertCopy                   1024          4  avgt   10   53,216 ±  3,361  ms/op
InsertBatch.insertCopy                   1024          8  avgt   10   27,275 ±  2,722  ms/op
InsertBatch.insertCopy                   1024         16  avgt   10   14,829 ±  2,795  ms/op
InsertBatch.insertCopy                   1024        128  avgt   10    2,835 ±  0,647  ms/op

@vlsi vlsi closed this in 510e6e0 Jun 22, 2016

@vlsi

This comment has been minimized.

Member

vlsi commented Jun 22, 2016

@Chrriis , many thanks for pushing this forward

@vlsi vlsi added this to the 9.4.1209 milestone Jun 22, 2016

zemian pushed a commit to zemian/pgjdbc that referenced this pull request Oct 6, 2016

fix: improve insert values(...) batch rewrite
Now it supports VALUES (?,1), and splits very long batch into series of smaller ones
to keep overall number of bind variables <32768 (that is PG's limit in v3 protocol)

Trigger property: reWriteBatchedInserts=true. Default value is still "false".

closes pgjdbc#580
closes pgjdbc#584
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment