feat: primitive arrays #887

bokken · 2017-07-27T21:23:10Z

Add support for primitive arrays.
This is a replacement for PR 872.

This is a minimal PR for purposes of discussion to support primitive arrays as part of issue #871 feat: minimal PR to support primitive arrays Improve StringBuilder init sizes. feat: support primitive arrays Add unit tests. Provide binary representation. Add javadoc. feat: support primitive arrays Use Oid rather than type name. More unit tests. feat: support primitive arrays Address checkstyle issues. feat: support primitive arrays Address checkstyle issues. Fix an issue with creating array string. Add support for boolean[]. Optimize mapping from String values to boolean. feat: support primitive arrays Address checkstyle issues. feat: support primitive arrays Use java 6 syntax. feat: support primitive arrays checkstyle in existing test feat: support primitive arrays checkstyle in existing test test: migrate tests to JUnit 4 fixes #738 fixes #205 test: assume minimum server version 8.3 testing autosave with ALTER Prior to 8.3, PostgreSQL didn't invalidate cached plans following DDL changes. This is mentioned in the 8.3 release notes as Automatically re-plan cached queries when table definitions change or statistics are updated https://www.postgresql.org/docs/8.3/static/release-8-3.html For details, see PostgreSQL commit b9527e9840 and friends. https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=b9527e984092e838790b543b014c0c2720ea4f11 test: assume minimum server version 8.3 when testing with uuid The UUID datatype was added to PostgreSQL in version 8.3. https://www.postgresql.org/docs/8.3/static/release-8-3.html refactor: remove unused import test: fix StringTypeParameterTest to skip preferQueryMode=simple Simple query mode does not send data type over the wire, so errors like "column \"m\" is of type mood but expression is of type character varying" never happen chore: install PostgreSQL 9.1 to Trusty builds via apt, and use Precise for Java 6 Travis deprecates Precise. See https://blog.travis-ci.com/2017-07-11-trusty-as-default-linux-is-coming sudo: required for dist: precise will still be supported for a while, however no updates to that image is planned Java 6 is not included into Trusty image, and installers no longer work (see http://www.webupd8.org/2017/06/why-oracle-java-7-and-6-installers-no.html) since Oracle removed the installer files from public access. refactor: remove useless checks in the tests remove useless checks for protocol version in the tests. style: update checkstyle + turn on some rules (#847) 1. Update checkstyle to 7.8.2 2. Add checkstyle module "RedundantModifier" 3. Uncomment checkstyle modules "ArrayTypeStyle", "ModifierOrder" test: make StringTypeParameterTest 8.3+ since 8.2 misses enum types (#882) PostgreSQL 8.2 tests at Travis should be green now, so removing allow_failures configuration test: assume integer datetimes for timestamp tests (#873) Float timestamp equality comparisons like comparisons with any float are problematic if they don't take into account their imprecise nature. Some timestamp tests produce false negative failures for servers compiled with float timestamps. Use JUnit assumptions to skip these tests if the server doesn't have integer datetimes. This addresses #12 in part in that it omits (most of) the failing tests, though it leaves these timestamp behaviors untested under float timestamps, which was the default prior to PostgreSQL 8.4. fix: named statements were used when fetchSize was non-zero and prepareThreshold=0 (#870) Non-zero fetchSize triggers use of named portals (for subsequent fetch requests), however named portals does not require to use named statements. As per PostgreSQL documentation, named portals are automatically closed as transaction completes, so named portals should play well with transaction-based poolers. fixes #869 test: skip ConcurrentStatementFetch for PostgreSQL < 8.4 (#884) PostgreSQL 8.2 is known to close named portals unexpectedly, so we just ignore the test. The following scenario might be affected: old backend version + setFetchSize + interleaved ResultSet processing This is a follow-up for #870 and #869 for PostgreSQL < 8.4 honour PGPORT, PGHOST, PGDBNAME in connection properties (#862) * honour PGPORT, PGHOST, PGDBNAME in connection properties feat: support primitive arrays use string array representation when simple query mode selected feat: support primitive arrays Checkstyle feat: support primitive arrays More unit test coverage feat: support primitive arrays Checkstyle test: migrate tests to Junit4 (#883) Migrate XATestSuite, Jdbc2TestSuite, Jdbc4TestSuite, ExtensionsTestSuite test suites see #205 feat: support primitive arrays Performance optimizations. Some renaming. feat: support primitive arrays Performance optimizations in TypeInfoCache. Checkstyle issue. feat: support primitive arrays Initialize maps in TypeInfoCache to appropriate size. feat: support primitive arrays Checkstyle feat: primitive arrays revert changes not directly related to supporting primitive arrays

codecov-io · 2017-07-27T22:05:09Z

Codecov Report

Merging #887 into master will increase coverage by 0.48%.
The diff coverage is 90.94%.

@@             Coverage Diff              @@
##             master     #887      +/-   ##
============================================
+ Coverage     66.35%   66.83%   +0.48%     
- Complexity     3607     3637      +30     
============================================
  Files           168      169       +1     
  Lines         15355    15593     +238     
  Branches       2483     2529      +46     
============================================
+ Hits          10189    10422     +233     
+ Misses         3985     3976       -9     
- Partials       1181     1195      +14

panchenko · 2017-07-28T03:36:04Z

pgjdbc/src/main/java/org/postgresql/jdbc/PrimitiveArraySupport.java

+  private static final Map<Class, PrimitiveArraySupport> ARRAY_CLASS_TOSTRING = new HashMap<Class, PrimitiveArraySupport>(9);
+
+  static {
+    ARRAY_CLASS_TOSTRING.put(long[].class, LONG_ARRAY_TOSTRING);


As binary serialization is also supported, I think the TOSTRING part can be removed from all the identifiers.

panchenko · 2017-07-28T04:09:19Z

pgjdbc/src/main/java/org/postgresql/jdbc/PgArray.java

+        Object array = readBinaryArray(1, 0);
+
+        if (PrimitiveArraySupport.isSupportedPrimitiveArray(array)) {
+          fieldString = PrimitiveArraySupport.arrayToString(connection.getTypeInfo().getArrayDelimiter(oid), array);


might be like this:

arraySupport = PrimitiveArraySupport.getArraySupport(array) if (arraySupport != null) { fieldString = arraySupport.toArrayString(array) }

The static PrimitiveArraySupport.arrayToString() method can be removed then.

Done.
The static utility arrayToString method was used more before binary support was added. I did not realize it had dropped to only 1 use when binary representation became preferred.

panchenko · 2017-07-28T04:16:36Z

pgjdbc/src/test/java/org/postgresql/test/jdbc4/ArrayTest.java

@@ -295,27 +313,34 @@ public void testUUIDArray() throws SQLException {
  @Test
  public void testSetObjectFromJavaArray() throws SQLException {
    String[] strArray = new String[]{"a", "b", "c"};
+    Object[] objCopy = new Object[strArray.length];
+    System.arraycopy(strArray, 0, objCopy, 0, strArray.length);


can be replaced with Arrays.copyOf(strArray, strArray.length, Object[].class)

Done.
Out of curiosity, why do you prefer the Arrays wrapper? These are functionally equivalent things, skipping some of the reflection stuff that Arrays has to do (or in this specific case because it is Object[], just a class identity comparison).

It's shorter, so more readable.

panchenko · 2017-07-28T04:23:40Z

pgjdbc/src/main/java/org/postgresql/PGArraySupport.java

+/**
+ * Defines public PostgreSQL extensions to create {@link Array} objects.
+ */
+public interface PGArraySupport {


How much this level of indirection is useful? How many methods can be here potentially? I would suggest declaring the method directly in PGConnection.

@vlsi seemed to want at least 1 more method.
It was not clear to me if it was just one more or several more.
I do not know at what number you would want to group together with a specific interface. I would think at 3 or so, but I do not really care either way.

This is a little bit closer to Connection if comparing with other extensions, as there is java.sql.Connection.createArrayOf() already.

code review comments

bokken · 2017-07-28T11:37:36Z

It still appears that the code coverage report is not including the new unit test class i added. How is it identifying what tests to run?

AlexElin · 2017-07-28T15:02:28Z

@bokken add your test class to the corresponding test suite

code review comments

documentation

bokken · 2017-08-16T11:49:48Z

@davecramer, @vlsi, @panchenko, is there anything else needed, or can this get merged in?

Thanks.

bokken · 2017-08-21T19:23:33Z

@vlsi and @davecramer any update on when this could get merged in?
I would like to be able to start using this without having to manage a fork.

davecramer · 2017-08-22T12:14:39Z

@bokken LGTM. @vlsi is on vacation. He should be back in a week

bokken · 2017-08-23T02:30:02Z

Thanks @davecramer

bokken · 2017-10-14T16:54:58Z

@vlsi, any update/eta on when this could get merged in?

bokken · 2017-11-20T19:21:22Z

@davecramer, @vlsi, any update/eta on when this could get merged in?

davecramer · 2017-11-21T11:43:56Z

@bokken Sorry, I'll commit to reviewing this on Wed. Thanks for your patience

vlsi · 2017-11-22T13:17:04Z

docs/documentation/head/arrays.md

+`int[]` | `int4[]`
+`long[]` | `int8[]`
+`float[]` | `float4[]`
+`double[]` | `float[]8`


float[]8 typo

bokken · 2017-12-12T14:05:05Z

@davecramer, @vlsi, any update/eta on when this could get merged in?

davecramer · 2017-12-13T12:11:24Z

@bokken there are some subtle semantic changes here. Specifically setObject(1, String[], Types.Array) used to fail, now it doesn't was this intentional?

bokken · 2017-12-13T16:17:39Z

@davecramer, yes that was intentional and is called out in the documentation:

https://github.com/pgjdbc/pgjdbc/pull/887/files?diff=unified#diff-b8b4949d263ba9fd44d157ae61de7584R19

The listed array types can go through setObject

vlsi · 2017-12-14T14:37:50Z

pgjdbc/src/main/java/org/postgresql/PGConnection.java

+   * @throws SQLException If for some reason the array cannot be created.
+   * @see java.sql.Connection#createArrayOf(String, Object[])
+   */
+  Array createArrayOf(String typeName, Object elements) throws SQLException;


Can Object elements be null?

No it cannot.
Similar to how createArrayOf(String typeName, Object[] elements) cannot take null elements either.

@bokken , here's Java 8 documentation:

https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html#createArrayOf-java.lang.String-java.lang.Object:A-

SQLException - if a database error occurs, the JDBC type is not appropriate for the typeName and the conversion is not supported, the typeName is null or this method is called on a closed connection

It does forbid null for typeName, however I do not see how it forbids null for elements

The added PGConnection.createArrayOf(String, Object) is a public API, and I think nullability documentation is a must.

PS. Sorry for the long-standing discussion, however we'd better be picky when adding new APIs.

I was speaking to the PGConnection implementation of createArrayOf, which also throws a NullPointerException if given null elements.
As this was not documented there, I did not document here.

https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/jdbc/PgConnection.java#L1149

Perhaps both methods should return null if elements is null?
I was simply expecting/providing symmetry with the existing implementation.

NullPointerException if given null elements might be an existing bug.

It appears that it is possible to create a PGArray representing a null array. Perhaps that is the solution?

So it is possible to create a null array in postgres so seems reasonable to allow it

select null::int[]; int4 ------

davecramer · 2017-12-15T10:41:50Z

@bokken can you fix the conflict and I'll commit this. Thanks

# Conflicts: # pgjdbc/src/test/java/org/postgresql/test/jdbc2/Jdbc2TestSuite.java

bokken · 2017-12-15T13:28:22Z

@davecramer, @vlsi,
It might be helpful to have an @since javadoc tag on the new createArrayOf method on PGConnection interface. What release would this be targeted for? 42.2?

I have fixed the conflict.

Let me know if you would like me to add the @since doc and/or make any changes to doc or implementation around handling of null elements.

vlsi · 2017-12-15T14:53:01Z

pgjdbc/src/main/java/org/postgresql/PGConnection.java

+
+  /**
+   * Creates an {@link Array} wrapping <i>elements</i>. This is similar to
+   * java.sql.Connection#createArrayOf(String, Object[]), but also provides


{@link ...createArrayOf...}

vlsi · 2017-12-15T14:53:31Z

pgjdbc/src/main/java/org/postgresql/PGConnection.java

+   * java.sql.Connection#createArrayOf(String, Object[]), but also provides
+   * support for primitive arrays.
+   *
+   * @param typeName


Please add nullability description

vlsi · 2017-12-15T14:53:42Z

pgjdbc/src/main/java/org/postgresql/PGConnection.java

+   *
+   * @param typeName
+   *          The SQL name of the type to map the <i>elements</i> to.
+   * @param elements


Please add nullability description

bokken · 2017-12-16T13:53:27Z

Which I think is effectively what current code does, right? It is simply wrapped in an Array object. For comparison, this seems to match ojdbc’s behavior.

…

On Sat, Dec 16, 2017 at 4:27 AM Dave Cramer ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In pgjdbc/src/main/java/org/postgresql/PGConnection.java <#887 (comment)>: > @@ -21,6 +22,22 @@ * returned by the PostgreSQL driver implement PGConnection. */ public interface PGConnection { + + /** + * Creates an ***@***.*** Array} wrapping elements. This is similar to + * java.sql.Connection# createArrayOf(String, Object[]), but also provides + * support for primitive arrays. + * + * @param typeName + * The SQL name of the type to map the elements to. + * @param elements + * The array of objects to map. + * @return An ***@***.*** Array} wrapping elements. + * @throws SQLException If for some reason the array cannot be created. + * @see java.sql.Connection#createArrayOf(String, Object[]) + */ + Array createArrayOf(String typeName, Object elements) throws SQLException; So it is possible to create a null array in postgres so seems reasonable to allow it select null::int[]; int4 ------ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#887 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC26bUiehoA_MmNpgDx0xr--J9atfHecks5tA5sPgaJpZM4Ol20_> .

bokken · 2017-12-19T11:49:30Z

@davecramer the conflict has been resolved.
@vlsi I believe I have covered all the doc updates you requested.
Null elements are now handled (with PGArray object wrapping null) for both the new method and the existing jdbc standard method.

…erly

BO8979 and others added 2 commits July 27, 2017 16:06

Merge remote-tracking branch 'upstream/master' into primitives

d660419

bokken mentioned this pull request Jul 27, 2017

feat: minimal PR to support primitive arrays #872

Closed

panchenko reviewed Jul 28, 2017

View reviewed changes

feat: primitive arrays

dd03051

code review comments

bokken added 2 commits July 28, 2017 11:31

feat: primitive arrays

9a18c64

code review comments

feat: primitive arrays

7303cb5

documentation

bokken mentioned this pull request Sep 8, 2017

BYTEA[] JDBC handling issue. #936

Closed

vlsi added the triage/needs-review Issue that needs a review - remove label if all is clear label Sep 25, 2017

vlsi reviewed Nov 22, 2017

View reviewed changes

doc typo

8007667

vlsi reviewed Dec 14, 2017

View reviewed changes

Merge remote-tracking branch 'upstream/master' into primitives

5050de2

# Conflicts: # pgjdbc/src/test/java/org/postgresql/test/jdbc2/Jdbc2TestSuite.java

bokken added 2 commits December 15, 2017 08:02

Handle creating array of null elements

c912ee5

remove trailing white spaces

d88a73b

vlsi reviewed Dec 15, 2017

View reviewed changes

bokken added 2 commits December 15, 2017 08:57

Update other ArrayTest to handle null arrays.

6dc4d7b

Document null values

f3dc162

docs: add Arrays section to the documenation, make tables render prop…

2f207e8

…erly

vlsi merged commit 3e0491a into pgjdbc:master Dec 19, 2017

vlsi removed the triage/needs-review Issue that needs a review - remove label if all is clear label Dec 19, 2017

vlsi added this to the 42.2.0 milestone Dec 19, 2017

pettyjamesm mentioned this pull request Dec 19, 2017

Use pgjdbc support for primitive arrays tminglei/slick-pg#380

Closed

bokken mentioned this pull request May 22, 2019

Support primitive arrays like String[], int[] in setObject(...) #19

Closed

jwgmeligmeyling mentioned this pull request Sep 15, 2019

HHH-10999 Array datatypes module hibernate/hibernate-orm#1499

Closed

davecramer mentioned this pull request Nov 13, 2019

Support JDK primitive arrays String[], int[] .. in setObject method #548

Closed

feat: primitive arrays #887

feat: primitive arrays #887

Conversation

bokken commented Jul 27, 2017

codecov-io commented Jul 27, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bokken commented Jul 28, 2017

AlexElin commented Jul 28, 2017

bokken commented Aug 16, 2017

bokken commented Aug 21, 2017

davecramer commented Aug 22, 2017

bokken commented Aug 23, 2017

bokken commented Oct 14, 2017

bokken commented Nov 20, 2017

davecramer commented Nov 21, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bokken commented Dec 12, 2017

davecramer commented Dec 13, 2017

bokken commented Dec 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davecramer commented Dec 15, 2017

bokken commented Dec 15, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bokken commented Dec 16, 2017 via email

bokken commented Dec 19, 2017

codecov-io commented Jul 27, 2017 •

edited

Loading