Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

42.2.5 poor performance with bulk copy #1312

Closed
niels1voo opened this issue Oct 18, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@niels1voo
Copy link

commented Oct 18, 2018

I'm submitting a ...

  • bug report
  • feature request

Describe the issue
Copy performance is very poor under driver version 42.2.5. We tested several versions prior to 42.2.5 and found them equivalent in performance. 42.2.5 was several orders of magnitude worse.

Java Version
1.8.0_181
OS Version
MacOS 10.14
PostgreSQL Version
9.6.10
To Reproduce
Steps to reproduce the behaviour:
(you may want to edit TestPedalDialect:205 to reduce the number of iterations, power 16 would suffice)
With version 42.2.4
clone https://github.com/niels1voo/pedal-dialect, modify JPAConfiguration lines 52-54 to suite your local db, run TestPedalDialect.testBulkCopyCommand.
notice the log info messages from CopyCommandImpl:90.
With 42.2.5
Edit pom.xml to change the postgresql dependency to version 42.2.5
Re-run TestPedalDialect.testBulkCopyCommand
Execution time slows by several orders of magnitude
Expected behaviour
42.2.5 should perform about as well as previous versions (500K inserts in 21 seconds)
And what actually happens
42.2.5 performs much, much worse than previous versions (500K inserts in 726 seconds)

Logs
pgjdbc-trace-42-2-4.log.gz
pgjdbc-trace-42-2-5.log.gz

@niels1voo niels1voo changed the title poor performanc with bulb copy poor performanc with bulk copy Oct 18, 2018

@niels1voo niels1voo changed the title poor performanc with bulk copy poor performance with bulk copy Oct 18, 2018

@niels1voo niels1voo changed the title poor performance with bulk copy 42.2.5 poor performance with bulk copy Oct 18, 2018

@davecramer

This comment has been minimized.

Copy link
Member

commented Oct 18, 2018

is there a simple way to run this with mvn?

@davecramer

This comment has been minimized.

Copy link
Member

commented Oct 18, 2018

running in intellij, gives me:
Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException

don't suppose you could make this simpler ?

@niels1voo

This comment has been minimized.

Copy link
Author

commented Oct 18, 2018

mvn -Dtest=TestPedalDialect#testBulkCopyCommand test
but you still need to setup the db per your test db preference.

@davecramer

This comment has been minimized.

Copy link
Member

commented Oct 18, 2018

OK, I can replicate this... thx

vlsi added a commit to vlsi/pgjdbc that referenced this issue Oct 18, 2018

perf: fix 1ms per async CopyAPI (regression since 42.2.5)
PGStream#hasMessagePending should not perform network requests otherwise it causes per row delay in async copy

fixes pgjdbc#1312

vlsi added a commit to vlsi/pgjdbc that referenced this issue Oct 19, 2018

perf: fix 1ms per async CopyAPI (regression since 42.2.5)
PGStream#hasMessagePending should not perform network requests otherwise it causes per row delay in async copy

fixes pgjdbc#1312

vlsi added a commit to vlsi/pgjdbc that referenced this issue Oct 21, 2018

perf: fix 1ms per async CopyAPI (regression since 42.2.5)
PGStream#hasMessagePending should not perform network requests otherwise it causes per row delay in async copy

fixes pgjdbc#1312

vlsi added a commit to vlsi/pgjdbc that referenced this issue Oct 21, 2018

perf: fix 1ms per async CopyAPI (regression since 42.2.5)
PGStream#hasMessagePending should not perform network requests otherwise it causes per row delay in async copy

fixes pgjdbc#1312

vlsi added a commit to vlsi/pgjdbc that referenced this issue Oct 24, 2018

perf: fix 1ms per async CopyAPI (regression since 42.2.5)
PGStream#hasMessagePending should not perform network requests otherwise it causes per row delay in async copy

fixes pgjdbc#1312

@vlsi vlsi closed this in e2623d6 Oct 25, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.