JVM crash Session causes SIGSEGV/SIGBUS under repeated use of Session.create()

[dfa1]

What happened?

There is a reproducer of the issue in
https://github.com/dfa1/datastore — a benchmark comparing CSV / JSON / Parquet / Vortex storage formats.

  git clone https://github.com/dfa1/datastore.git
  cd datastore
  mvn test -Dtest="ReadColumnTest,VortexOhlcStoreTest"  # passes (few iterations)
  mvn test -Dtest="BenchmarkTest"                        # crashes (84+ Vortex ops)

BenchmarkTest runs 7 scales × (2 warmup + 10 measured) iterations × 3 operations (write / read /
readColumn) = 252 Session objects created in one JVM process.

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00000001393da064, pid=71272, tid=5635
#
# JRE version: OpenJDK Runtime Environment (27.0+1) (build 27-jep401ea3+1-1)
# Java VM: OpenJDK 64-Bit Server VM (27-jep401ea3+1-1, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
# Problematic frame:
# C  [libvortex_jni9112159322116037794.dylib+0x32064]  vortex_file::writer::WriteOptionsSessionExt::write_options::h986f424532314d6f+0x18
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

Possible fix:

Make Session implement AutoCloseable and expose the Cleanable so callers can release native deterministically:

  public final class Session implements AutoCloseable {
      private final Cleaner.Cleanable cleanable;

      private Session(long pointer) {
          Preconditions.checkArgument(pointer != 0, "invalid session pointer");
          this.pointer   = pointer;
          this.cleanable = VortexCleaner.register(this, () -> NativeSession.free(pointer));
      }

      @Override
      public void close() {
          cleanable.clean(); // idempotent — safe to call; Cleaner won't double-free
      }
  }

Callers could then write:

  try (Session session = Session.create()) {
      // ...
  }

Steps to reproduce

see the attached instructions above

Environment

• vortex-jni version: 0.72.0
• JDK 27 EA (Homebrew, aarch64 macOS): crashes ✗ — SIGSEGV/SIGBUS under repeated Session use
• Arch: bsd-aarch64

Additional context

I got the error also once in JDK 25 but it is much harder to reproduce.
The proposed fix is the idiomatic way in Java to deal with external resources.

[dfa1]

NativeSession.free is also exposed as public method: I tried to use it by I got a double free (because free is also called by the cleaner).
IMHO it should be marked as package private (not part of the public API at least)

[dfa1]

hs_err_pid66301.log
Here is the full log, interesting part is:

Current thread (0x0000000104f3cfd0):  JavaThread "main"             [_thread_in_native, id=6147, stack(0x000000016b624000,0x000000016b827000) (2060K)]

Stack: [0x000000016b624000,0x000000016b827000],  sp=0x000000016b822040,  free space=2040k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libvortex_jni5295561805888454796.dylib+0x32064]  vortex_file::writer::WriteOptionsSessionExt::write_options::h986f424532314d6f+0x18
C  [libvortex_jni5295561805888454796.dylib+0x264d0]  vortex_jni::writer::Java_dev_vortex_jni_NativeWriter_create::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h84fc0ede4f6ff94b+0xb4c
C  [libvortex_jni5295561805888454796.dylib+0x71c70]  _$LT$tracing..instrument..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::he6d38967d6ba0ad1+0xf0
C  [libvortex_jni5295561805888454796.dylib+0x55e658]  async_task::raw::RawTask$LT$F$C$T$C$S$C$M$GT$::run::h7f6d87ee8210d49c+0x88
C  [libvortex_jni5295561805888454796.dylib+0x135b8]  Java_dev_vortex_jni_NativeWriter_close+0x698
j  dev.vortex.jni.NativeWriter.close(J)V+0
J 19933 c2 io.github.dfa1.datastore.VortexOhlcStore.write(Ljava/util/stream/Stream;Ljava/nio/file/Path;)V (170 bytes) @ 0x000000011c0c2914 [0x000000011c0c1d00+0x0000000000000c14]
j  io.github.dfa1.datastore.BenchmarkTest.measure(Lio/github/dfa1/datastore/OhlcStore;Ljava/util/List;Ljava/nio/file/Path;)Lio/github/dfa1/datastore/BenchmarkTest$Result;+110
j  io.github.dfa1.datastore.BenchmarkTest.lambda$storageComparison$0(Ljava/util/List;Ljava/nio/file/Path;Lio/github/dfa1/datastore/OhlcStore;)Lio/github/dfa1/datastore/BenchmarkTest$Result;+4

[dfa1]

More minimal reproducer: this fails after 200-300 iterations on JDK 26 and about 18K on JDK 25

	@Test
	void reproducer(@TempDir Path dir) throws IOException {
		VortexOhlcStore store = new VortexOhlcStore();
		int i = 0;
		while (true) {
			System.out.println(i);
			OhlcGenerator aaa = new OhlcGenerator(new Symbol("AAA"), LocalDate.now(), 10, 10);
			store.write(aaa.stream(10), dir.resolve("a" + i++));
		}
	}

[a10y]

Hey @dfa1, I was able to reproduce the failure from your last example. Thanks for including a compact repro.

Interestingly, moving the Session object in the OhlcStore write method to be a static or instance-var prevents the failure.

I agree that the Session should be AutoCloseable.

Separately, I'd recommend that you update the Vortex store to hold a long-lived Session object. Creating new Sessions can be expensive, generally they should be initialized statically and shared across a process.

[dfa1]

Thanks @a10y for feedback! I moved Session to a private static final field so it's created once and shared across all operations.

This fixes the crash I reported, but I'm now seeing a different failure when the same session is reused across multiple sequential writes + reads on different file paths:

java.io.IOException: [errno 22] CDataJniException{errno=22, message=External error: SerializedArray

buffer is too short for flatbuffer}
at org.apache.arrow.c.ArrowArrayStream.getNext(ArrowArrayStream.java:146)
at org.apache.arrow.c.ArrowArrayStreamReader.loadNextBatch(ArrowArrayStreamReader.java:63)

This was the reason why I started to create a new Session per call.

Repro pattern (no concurrency, single thread):

    private static final Session SESSION = Session.create();

    // iteration 1: ~250 rows → succeeds
    VortexWriter.create(SESSION, uri1, schema, opts, allocator);  // write
    DataSource.open(SESSION, uri1).scan(...);                      // read OK

    // iteration 2: ~2500 rows → read throws errno=22
    VortexWriter.create(SESSION, uri2, schema, opts, allocator);  // write
    DataSource.open(SESSION, uri2).scan(...);                      // FAIL

The first iteration (small file) succeeds. The second iteration (larger file, different URI) fails
during loadNextBatch. There is a new reproducer here:

https://github.com/dfa1/datastore/blob/main/src/test/java/io/github/dfa1/datastore/VortexSessionReuseReproTest.java

Perhaps I'm using the API in the wrong way?