fix: binary processing of Date/Time/Timestamps

[vlsi]

This is a follow-up on #376.
I force TimezoneTest to test both text and binary encoding and it makes

It looks like this PR fixes all known problems with Timestamp/Date/Time parsing.
I wish someone could add more tests.

Here are the measurements (jdk 1.8u60, OS X 10.11, Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz, time zone == Europe/Moscow, local postgresql 9.4).
The test is to perform and fully fetch a query like select localtimestamp c0, localtimestamp c1, ... from generate_series(1, ?) as t(x): https://github.com/pgjdbc/pgjdbc/pull/387/files#diff-962be0de5c547f8cb43ea8efd768dccaR88

Notes:

1. There is a slight degradation of getTimestamp when selecting localtimestamp if compared with 1203. I believe the degradation is due to more complex logic in TimestampUtil.guessTimestamp.
   I believe it is not a showstopper as I see no alternative solution there: https://github.com/pgjdbc/pgjdbc/pull/387/files#diff-d3ce26f4020b4504cf0a8ef2e019b316R699
2. If timezone is provided by backend (or if is requested as GMT+-XX:...), the performance is better: 130us vs 350us for 100 rows x 5 columns test. No degradation here if compared with 1203
3. BIGINT is somewhat slow. The case is getBigDecimal for 1234567890123456789
4. 1201 is the slowest in all the cases
5. Caching of TimeZone.getDefault() does not buy much, thus I switched to TimeZone.getDefault() when needed. So pgjdbc supports "default timezone" changes.
6. pgjdbc is somewhat slower than pgjdbc-ng for receiving BIGINT and sometimes TIMESTAMP. Otherwise, pgjdbc is like twice as fast as pgjdbc-ng.

With current fix:

Benchmark                          (ncols)  (nrows)       (type)  Mode  Cnt    Score   Error  Units
ProcessResultSet.bindExecuteFetch        5        1          INT  avgt   10   44,268 ± 1,090  us/op
ProcessResultSet.bindExecuteFetch        5        1       BIGINT  avgt   10   47,229 ± 0,400  us/op
ProcessResultSet.bindExecuteFetch        5        1       STRING  avgt   10   45,106 ± 0,541  us/op
ProcessResultSet.bindExecuteFetch        5        1    TIMESTAMP  avgt   10   50,908 ± 0,930  us/op
ProcessResultSet.bindExecuteFetch        5        1  TIMESTAMPTZ  avgt   10   46,002 ± 0,815  us/op
ProcessResultSet.bindExecuteFetch        5       50          INT  avgt   10   77,516 ± 0,771  us/op
ProcessResultSet.bindExecuteFetch        5       50       BIGINT  avgt   10  210,191 ± 2,177  us/op
ProcessResultSet.bindExecuteFetch        5       50       STRING  avgt   10   87,583 ± 1,627  us/op
ProcessResultSet.bindExecuteFetch        5       50    TIMESTAMP  avgt   10  189,531 ± 1,372  us/op
ProcessResultSet.bindExecuteFetch        5       50  TIMESTAMPTZ  avgt   10   86,771 ± 0,690  us/op
ProcessResultSet.bindExecuteFetch        5      100          INT  avgt   10  114,029 ± 1,062  us/op
ProcessResultSet.bindExecuteFetch        5      100       BIGINT  avgt   10  398,847 ± 4,706  us/op
ProcessResultSet.bindExecuteFetch        5      100       STRING  avgt   10  123,094 ± 0,944  us/op
ProcessResultSet.bindExecuteFetch        5      100    TIMESTAMP  avgt   10  360,408 ± 4,311  us/op
ProcessResultSet.bindExecuteFetch        5      100  TIMESTAMPTZ  avgt   10  131,307 ± 1,544  us/op

With caching of TimeZone.getDefault() (like in previous pgjdbc versions):

Benchmark                          (ncols)  (nrows)       (type)  Mode  Cnt    Score   Error  Units
ProcessResultSet.bindExecuteFetch        5        1    TIMESTAMP  avgt   10   48,748 ± 0,423  us/op
ProcessResultSet.bindExecuteFetch        5        1  TIMESTAMPTZ  avgt   10   46,089 ± 0,491  us/op
ProcessResultSet.bindExecuteFetch        5       50    TIMESTAMP  avgt   10  187,591 ± 1,812  us/op
ProcessResultSet.bindExecuteFetch        5       50  TIMESTAMPTZ  avgt   10   86,319 ± 0,990  us/op
ProcessResultSet.bindExecuteFetch        5      100    TIMESTAMP  avgt   10  353,440 ± 4,387  us/op
ProcessResultSet.bindExecuteFetch        5      100  TIMESTAMPTZ  avgt   10  131,478 ± 1,669  us/op

1203 (2014-09-18, a0bafcd):

Benchmark                          (ncols)  (nrows)       (type)  Mode  Cnt    Score   Error  Units
ProcessResultSet.bindExecuteFetch        5        1          INT  avgt   10   43,859 ± 0,304  us/op
ProcessResultSet.bindExecuteFetch        5        1       BIGINT  avgt   10   47,499 ± 0,737  us/op
ProcessResultSet.bindExecuteFetch        5        1       STRING  avgt   10   44,528 ± 0,820  us/op
ProcessResultSet.bindExecuteFetch        5        1    TIMESTAMP  avgt   10   48,178 ± 0,493  us/op
ProcessResultSet.bindExecuteFetch        5        1  TIMESTAMPTZ  avgt   10   45,336 ± 0,544  us/op
ProcessResultSet.bindExecuteFetch        5       50          INT  avgt   10   76,734 ± 0,767  us/op
ProcessResultSet.bindExecuteFetch        5       50       BIGINT  avgt   10  208,770 ± 2,250  us/op
ProcessResultSet.bindExecuteFetch        5       50       STRING  avgt   10   87,040 ± 0,606  us/op
ProcessResultSet.bindExecuteFetch        5       50    TIMESTAMP  avgt   10  149,451 ± 1,147  us/op
ProcessResultSet.bindExecuteFetch        5       50  TIMESTAMPTZ  avgt   10   86,390 ± 0,796  us/op
ProcessResultSet.bindExecuteFetch        5      100          INT  avgt   10  112,417 ± 1,183  us/op
ProcessResultSet.bindExecuteFetch        5      100       BIGINT  avgt   10  390,190 ± 5,070  us/op
ProcessResultSet.bindExecuteFetch        5      100       STRING  avgt   10  123,966 ± 1,410  us/op
ProcessResultSet.bindExecuteFetch        5      100    TIMESTAMP  avgt   10  263,847 ± 3,001  us/op
ProcessResultSet.bindExecuteFetch        5      100  TIMESTAMPTZ  avgt   10  131,515 ± 2,472  us/op

1201 (2015-02-28, a4f8c9e):

Benchmark                          (ncols)  (nrows)       (type)  Mode  Cnt    Score   Error  Units
ProcessResultSet.bindExecuteFetch        5        1          INT  avgt   10   81,928 ± 2,433  us/op
ProcessResultSet.bindExecuteFetch        5        1       BIGINT  avgt   10   84,142 ± 1,656  us/op
ProcessResultSet.bindExecuteFetch        5        1       STRING  avgt   10   81,131 ± 0,537  us/op
ProcessResultSet.bindExecuteFetch        5        1    TIMESTAMP  avgt   10  110,941 ± 1,069  us/op
ProcessResultSet.bindExecuteFetch        5        1  TIMESTAMPTZ  avgt   10   94,211 ± 1,269  us/op
ProcessResultSet.bindExecuteFetch        5       50          INT  avgt   10  118,746 ± 1,309  us/op
ProcessResultSet.bindExecuteFetch        5       50       BIGINT  avgt   10  204,295 ± 2,628  us/op
ProcessResultSet.bindExecuteFetch        5       50       STRING  avgt   10  131,523 ± 3,601  us/op
ProcessResultSet.bindExecuteFetch        5       50    TIMESTAMP  avgt   10  446,562 ± 4,103  us/op
ProcessResultSet.bindExecuteFetch        5       50  TIMESTAMPTZ  avgt   10  396,859 ± 6,574  us/op
ProcessResultSet.bindExecuteFetch        5      100          INT  avgt   10  156,238 ± 1,413  us/op
ProcessResultSet.bindExecuteFetch        5      100       BIGINT  avgt   10  289,011 ± 2,651  us/op
ProcessResultSet.bindExecuteFetch        5      100       STRING  avgt   10  157,809 ± 1,809  us/op
ProcessResultSet.bindExecuteFetch        5      100    TIMESTAMP  avgt   10  730,901 ± 7,670  us/op
ProcessResultSet.bindExecuteFetch        5      100  TIMESTAMPTZ  avgt   10  677,107 ± 5,350  us/op

pgjdbc-ng 0.6-SNAPSHOT (2015-10-09, impossibl/pgjdbc-ng@1f8cdcb)

Benchmark                          (ncols)  (nrows)       (type)  Mode  Cnt    Score    Error  Units
ProcessResultSet.bindExecuteFetch        5        1          INT  avgt   10  107,731 ±  1,419  us/op
ProcessResultSet.bindExecuteFetch        5        1       BIGINT  avgt   10  108,796 ±  1,502  us/op
ProcessResultSet.bindExecuteFetch        5        1       STRING  avgt   10  107,165 ±  1,532  us/op
ProcessResultSet.bindExecuteFetch        5        1    TIMESTAMP  avgt   10  112,654 ±  1,862  us/op
ProcessResultSet.bindExecuteFetch        5        1  TIMESTAMPTZ  avgt   10  111,331 ±  5,782  us/op
ProcessResultSet.bindExecuteFetch        5       50          INT  avgt   10  145,734 ±  2,173  us/op
ProcessResultSet.bindExecuteFetch        5       50       BIGINT  avgt   10  209,057 ±  2,261  us/op
ProcessResultSet.bindExecuteFetch        5       50       STRING  avgt   10  167,522 ±  2,010  us/op
ProcessResultSet.bindExecuteFetch        5       50    TIMESTAMP  avgt   10  276,840 ±  5,459  us/op
ProcessResultSet.bindExecuteFetch        5       50  TIMESTAMPTZ  avgt   10  219,972 ±  1,909  us/op
ProcessResultSet.bindExecuteFetch        5      100          INT  avgt   10  187,370 ±  4,903  us/op
ProcessResultSet.bindExecuteFetch        5      100       BIGINT  avgt   10  302,751 ±  3,111  us/op
ProcessResultSet.bindExecuteFetch        5      100       STRING  avgt   10  214,831 ±  3,807  us/op
ProcessResultSet.bindExecuteFetch        5      100    TIMESTAMP  avgt   10  477,054 ± 10,552  us/op
ProcessResultSet.bindExecuteFetch        5      100  TIMESTAMPTZ  avgt   10  329,831 ±  2,572  us/op

pgjdbc-ng 0.6, "do not close preparedstatement test"

Benchmark                          (ncols)  (nrows)       (type)  Mode  Cnt    Score   Error  Units
ProcessResultSet.bindExecuteFetch        5        1          INT  avgt   10   83,651 ± 0,619  us/op
ProcessResultSet.bindExecuteFetch        5        1       BIGINT  avgt   10   85,302 ± 1,135  us/op
ProcessResultSet.bindExecuteFetch        5        1       STRING  avgt   10   84,268 ± 1,074  us/op
ProcessResultSet.bindExecuteFetch        5        1    TIMESTAMP  avgt   10   92,074 ± 1,756  us/op
ProcessResultSet.bindExecuteFetch        5        1  TIMESTAMPTZ  avgt   10   89,613 ± 3,663  us/op
ProcessResultSet.bindExecuteFetch        5       50          INT  avgt   10  122,788 ± 2,616  us/op
ProcessResultSet.bindExecuteFetch        5       50       BIGINT  avgt   10  183,256 ± 1,257  us/op
ProcessResultSet.bindExecuteFetch        5       50       STRING  avgt   10  143,807 ± 2,142  us/op
ProcessResultSet.bindExecuteFetch        5       50    TIMESTAMP  avgt   10  251,899 ± 3,357  us/op
ProcessResultSet.bindExecuteFetch        5       50  TIMESTAMPTZ  avgt   10  196,308 ± 1,799  us/op
ProcessResultSet.bindExecuteFetch        5      100          INT  avgt   10  161,394 ± 1,482  us/op
ProcessResultSet.bindExecuteFetch        5      100       BIGINT  avgt   10  277,571 ± 3,665  us/op
ProcessResultSet.bindExecuteFetch        5      100       STRING  avgt   10  191,090 ± 5,264  us/op
ProcessResultSet.bindExecuteFetch        5      100    TIMESTAMP  avgt   10  444,816 ± 4,581  us/op
ProcessResultSet.bindExecuteFetch        5      100  TIMESTAMPTZ  avgt   10  306,673 ± 2,182  us/op

pgjdbc-ng 0.5 (2015-04-30, impossibl/pgjdbc-ng@9cefd79)

Benchmark                          (ncols)  (nrows)       (type)  Mode  Cnt    Score   Error  Units
ProcessResultSet.bindExecuteFetch        5        1          INT  avgt   10  106,952 ± 1,894  us/op
ProcessResultSet.bindExecuteFetch        5        1       BIGINT  avgt   10  109,359 ± 3,502  us/op
ProcessResultSet.bindExecuteFetch        5        1       STRING  avgt   10  108,511 ± 2,270  us/op
ProcessResultSet.bindExecuteFetch        5        1    TIMESTAMP  avgt   10  112,952 ± 2,138  us/op
ProcessResultSet.bindExecuteFetch        5        1  TIMESTAMPTZ  avgt   10  109,572 ± 2,632  us/op
ProcessResultSet.bindExecuteFetch        5       50          INT  avgt   10  150,396 ± 6,917  us/op
ProcessResultSet.bindExecuteFetch        5       50       BIGINT  avgt   10  205,146 ± 2,679  us/op
ProcessResultSet.bindExecuteFetch        5       50       STRING  avgt   10  168,976 ± 2,761  us/op
ProcessResultSet.bindExecuteFetch        5       50    TIMESTAMP  avgt   10  277,680 ± 3,604  us/op
ProcessResultSet.bindExecuteFetch        5       50  TIMESTAMPTZ  avgt   10  221,149 ± 2,832  us/op
ProcessResultSet.bindExecuteFetch        5      100          INT  avgt   10  187,435 ± 2,364  us/op
ProcessResultSet.bindExecuteFetch        5      100       BIGINT  avgt   10  298,107 ± 2,482  us/op
ProcessResultSet.bindExecuteFetch        5      100       STRING  avgt   10  217,826 ± 1,990  us/op
ProcessResultSet.bindExecuteFetch        5      100    TIMESTAMP  avgt   10  476,290 ± 7,048  us/op
ProcessResultSet.bindExecuteFetch        5      100  TIMESTAMPTZ  avgt   10  330,508 ± 3,006  us/op

[davecramer]

interestingly this travis-ci is having issues with this

[vlsi]

interestingly this travis-ci is having issues with this

It does not. I'm a bit lazy and I do not always run full test suite.
There was no a single failure that I was not able to reproduce locally.

I expect 1fe0255466154150c01637a901acfc72ca3f6689 to pass in Travis.

[vlsi]

I did alter previous behavior a bit:
https://github.com/pgjdbc/pgjdbc/pull/387/files#diff-3c7c6e6379cec503322d6c309d14b3bdR172
https://github.com/pgjdbc/pgjdbc/pull/387/files#diff-3c7c6e6379cec503322d6c309d14b3bdR312

In two words: when backend responds with time or time with time zone type, I make sure "date part" of java timestamp (no matter if it is Timestamp or Time) is set to 1970-01-01.
Previously, certain timestamps in certain timezones might result into timestamps like "1970-01-02 HH:MM" (the 2nd of January).

[davecramer]

No, that's not the issue. It is taking a very long time to run. Much longer
than I would expect

Dave Cramer

On 8 October 2015 at 08:22, Vladimir Sitnikov notifications@github.com
wrote:

interestingly this travis-ci is having issues with this

It does not. I'm a bit lazy and I do not always run full test suite.
There was no a single failure that I was not able to reproduce locally.

I expect 1fe0255
1fe0255
to pass in Travis.

—
Reply to this email directly or view it on GitHub
#387 (comment).

[davecramer]

Is this in one of the tests ?

Dave Cramer

On 7 October 2015 at 17:45, Vladimir Sitnikov notifications@github.com
wrote:

Observation 1 is TimeZone.setDefault(TimeZone.getTimeZone("GMT+01")); is
weird when combined with con.createStatement().executeUpdate("set
timezone = 'gmt-3'");

This causes nasty things like "backend returning timestamps as +03:00
strings" while driver trying to use "+01:00".

This messes things up.

So the question is: how should "getString/setString" be supported in case
of non-matching zone settings.

When parsing server-side response, we can figure out that data type is
"timestamp with time zone", so we can return "proper value as if it were in
java default time zone", so it looks like getString(i) should be patched
to fixup string representation of date/timestamp in question.

—
Reply to this email directly or view it on GitHub
#387 (comment).

[vlsi]

Is this in one of the tests ?

The test was already there, however I altered "expected" value.

[davecramer]

OK, so to answer what I think your question is.

If you use getString that implies no Calendar. Without a calendar we have
to use the JVM default calendar to construct the Timestamp/date/time

Dave Cramer

On 8 October 2015 at 08:36, Vladimir Sitnikov notifications@github.com
wrote:

Is this in one of the tests ?

The test was already there, however I altered "expected" value.

—
Reply to this email directly or view it on GitHub
#387 (comment).

[vlsi]

OK, so to answer what I think your question is.

I think you answer to some other question.

My current question is: what are the missing tests.
If 1fe0255 is fine from functional point of view, it requires some benchmarking.

[davecramer]

Comments about why we are adding the offset in before zeroing out the time
would be appreciated; as well as anywhere else where you are manipulating
the time

Dave Cramer

On 8 October 2015 at 08:45, Vladimir Sitnikov notifications@github.com
wrote:

OK, so to answer what I think your question is.

I think you answer to some other question.

My current question is: what are the missing tests.
If 1fe0255
1fe0255
is fine from functional point of view, it requires some benchmarking.

—
Reply to this email directly or view it on GitHub
#387 (comment).

[vlsi]

I've updated implementation, added comments, added more tests.

Please review.