[Java] "Buffer too large to resize to ..." error

[asfimport]

On the latest branch for ARROW-394 (after rebasing on all the fixes so far), I have the error

Incompatible files
Could not load buffers for field f2: Utf8. error message: Buffer too large to resize to 44: 128
13:41:19.356 [main] ERROR org.apache.arrow.tools.Integration - Incompatible files
java.lang.IllegalArgumentException: Could not load buffers for field f2: Utf8. error message: Buffer too large to resize to 44: 128
	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:84) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:94) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.VectorLoader.load(VectorLoader.java:63) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.tools.Integration$Command$3.execute(Integration.java:156) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.tools.Integration.run(Integration.java:212) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.tools.Integration.main(Integration.java:61) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
Caused by: java.lang.IllegalArgumentException: Buffer too large to resize to 44: 128
	at org.apache.arrow.vector.BaseDataValueVector.truncateBufferBasedOnSize(BaseDataValueVector.java:55) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.NullableVarCharVector.loadFieldBuffers(NullableVarCharVector.java:142) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.apache.arrow.vector.VectorLoader.loadBuffers(VectorLoader.java:82) ~[arrow-tools-0.1.1-SNAPSHOT-jar-with-dependencies.jar:na]
	... 5 common frames omitted

See attached JSON and C++-generated Arrow file

Reporter: Wes McKinney / @wesm
Assignee: Wes McKinney / @wesm

Original Issue Attachments:

• struct_example.arrow
• struct_example.json

_{Note: This issue was originally created as ARROW-414. Please see the migration documentation for further details.}

[asfimport]

Wes McKinney / @wesm:
Seems related to ARROW-412

[asfimport]

Julien Le Dem / @julienledem:
So it's complaining that the padding is more than 64 bytes.
based on fieldNode.getLength() the buffer should be of length 44 ((10 values + 1) x 4 bytes)
I added a check that padding should be no more than 64 bytes so 128 is too big.
(and I guess I should verify that it is on 64 bytes boundaries instead of < 64)

Is the padding rule correct?
How strict do we want to be?
In general I prefer to be stricter to catch bugs.

[asfimport]

Wes McKinney / @wesm:
Resolved (for now) in:

45ed7e7