Skip to content

Faster field lookups by eliminating looping on schema field lookups for field names during indexing #20121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bharath-techie merged 1 commit into from
Dec 4, 2025
+8 −1
Merged

Faster field lookups by eliminating looping on schema field lookups for field names during indexing #20121

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion modules/parquet-data-format/src/main/java/com/parquet/parquetdataformat/vsr/ManagedVSR.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,14 @@
import org.apache.arrow.memory.RootAllocator;
import org.apache.arrow.vector.BigIntVector;
import org.apache.arrow.vector.VectorSchemaRoot;
import org.apache.arrow.vector.types.pojo.Field;
import org.apache.arrow.vector.FieldVector;
import org.apache.arrow.c.ArrowArray;
import org.apache.arrow.c.ArrowSchema;
import org.apache.arrow.c.Data;

import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
Expand All @@ -29,6 +32,7 @@ public class ManagedVSR implements AutoCloseable {
private final AtomicReference<VSRState> state;
private final ReadWriteLock lock;
private final long createdTime;
private final Map<String, Field> fields = new HashMap<>();


public ManagedVSR(String id, VectorSchemaRoot vsr, BufferAllocator allocator) {
Expand All @@ -38,6 +42,9 @@ public ManagedVSR(String id, VectorSchemaRoot vsr, BufferAllocator allocator) {
this.state = new AtomicReference<>(VSRState.ACTIVE);
this.lock = new ReentrantReadWriteLock();
this.createdTime = System.currentTimeMillis();
for (Field field : vsr.getSchema().getFields()) {
fields.put(field.getName(), field);
}
}

/**
Expand Down Expand Up @@ -94,7 +101,7 @@ public void setRowCount(int rowCount) {
public FieldVector getVector(String fieldName) {
lock.readLock().lock();
try {
return vsr.getVector(fieldName);
return vsr.getVector(fields.get(fieldName));
} finally {
lock.readLock().unlock();
}
Expand Down
Loading