Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java]: Expose util methods inside VectorAppender class such as append vector values #37246

Closed
davisusanibar opened this issue Aug 18, 2023 · 0 comments · Fixed by #37247
Closed

Comments

@davisusanibar
Copy link
Contributor

Describe the enhancement requested

Current VectorAppender class is package-private.

This class provides utility methods for appending vector values fields that can be exposed publicly.

Java code:

    package org.apache.arrow.vector.util;
    import org.apache.arrow.memory.BufferAllocator;
    import org.apache.arrow.memory.RootAllocator;
    import org.apache.arrow.vector.IntVector;
    import org.apache.arrow.vector.ValueVector;
    public class FieldVectorAppender {
      public static void main(String[] args) {
        try (
            BufferAllocator allocator = new RootAllocator();
            IntVector initialValues = new IntVector("initialValues", allocator);
            IntVector toAppend = new IntVector("toAppend", allocator);
        ) {
          initialValues.allocateNew(2);
          initialValues.set(0, 1);
          initialValues.set(1, 2);
          initialValues.setValueCount(2);
          System.out.println("Initial IntVector: " + initialValues);
          toAppend.allocateNew(4);
          toAppend.set(1, 4);
          toAppend.set(3, 6);
          toAppend.setValueCount(4);
          System.out.println("IntVector to Append: " + toAppend);
          VectorAppender appenderUtil = new VectorAppender(initialValues);
          ValueVector resultOfVectorsAppended = toAppend.accept(appenderUtil, null);
          System.out.println("IntVector Result: " + resultOfVectorsAppended);
        }
      }
    }

Result:

    Initial IntVector: [1, 2]
    IntVector to Append: [null, 4, null, 6]
    IntVector Result: [1, 2, null, 4, null, 6]

Component(s)

Java

lidavidm pushed a commit that referenced this issue Aug 18, 2023
…nd vector values (#37247)

### Rationale for this change

- To close #37246 

### What changes are included in this PR?

- Expose VectorAppender class to offer support to append vector values

### Are these changes tested?

- By TestVectorAppender

### Are there any user-facing changes?

- No
* Closes: #37246

Authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
@lidavidm lidavidm added this to the 14.0.0 milestone Aug 18, 2023
dongjoon-hyun pushed a commit to apache/spark that referenced this issue Nov 4, 2023
### What changes were proposed in this pull request?
This pr upgrade Apache Arrow from 13.0.0 to 14.0.0.

### Why are the changes needed?
The Apache Arrow 14.0.0 release brings a number of enhancements and bug fixes.
‎
In terms of bug fixes, the release addresses several critical issues that were causing failures in integration jobs with Spark([GH-36332](apache/arrow#36332)) and problems with importing empty data arrays([GH-37056](apache/arrow#37056)). It also optimizes the process of appending variable length vectors([GH-37829](apache/arrow#37829)) and includes C++ libraries for MacOS AARCH 64 in Java-Jars([GH-38076](apache/arrow#38076)).
‎
The new features and improvements focus on enhancing the handling and manipulation of data. This includes the introduction of DefaultVectorComparators for large types([GH-25659](apache/arrow#25659)), support for extended expressions in ScannerBuilder([GH-34252](apache/arrow#34252)), and the exposure of the VectorAppender class([GH-37246](apache/arrow#37246)).
‎
The release also brings enhancements to the development and testing process, with the CI environment now using JDK 21([GH-36994](apache/arrow#36994)). In addition, the release introduces vector validation consistent with C++, ensuring consistency across different languages([GH-37702](apache/arrow#37702)).
‎
Furthermore, the usability of VarChar writers and binary writers has been improved with the addition of extra input methods([GH-37705](apache/arrow#37705)), and VarCharWriter now supports writing from `Text` and `String`([GH-37706](apache/arrow#37706)). The release also adds typed getters for StructVector, improving the ease of accessing data([GH-37863](apache/arrow#37863)).

The full release notes as follows:
- https://arrow.apache.org/release/14.0.0.html

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #43650 from LuciferYang/arrow-14.

Lead-authored-by: yangjie01 <yangjie01@baidu.com>
Co-authored-by: YangJie <yangjie01@baidu.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this issue Nov 13, 2023
…o append vector values (apache#37247)

### Rationale for this change

- To close apache#37246 

### What changes are included in this PR?

- Expose VectorAppender class to offer support to append vector values

### Are these changes tested?

- By TestVectorAppender

### Are there any user-facing changes?

- No
* Closes: apache#37246

Authored-by: david dali susanibar arce <davi.sarces@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants