Apache Parquet 1.18.0 RC1

What's Changed

• MINOR: Fix error message in SimpleGroup.add() for Binary by @takezoe in #3385
• [minor] remove comment from pom.xml by @dossett in #3382
• Minor: ci-before_install.sh should show wget errors and retry by @dossett in #3381
• Minor: Upgrade spotless plugin by @dossett in #3380
• [minor] Move on from java8 for three core modules by @dossett in #3379
• MINOR: japicmp checks are failing after Java 11 release by @dossett in #3388
• MINOR: Bump version to 1.18.0-SNAPSHOT by @wgtmac in #3389
• GH-3372: Enhance ColumnSizeCommand to sort by size by @wgtmac in #3371
• GH-3399: Hint IDE the source code generated by thrift by @HuaHuaY in #3400
• GH-3387: bump aircompressor to mitigate CVE-2025-67721 by @olegbonar in #3403
• GH-3383: Modernize mockito by @dossett in #3384
• GH-3404: fix CI TestByteBitPacking512VectorLE.unpackValuesUsingVector randomly oom by @HuaHuaY in #3405
• GH-3401: Remove redundant MAP_KEY_VALUE annotation at middle level of logical type map by @HuaHuaY in #3402
• GH-3406: fix Float16 statistics handling for NaN and zero values by @HuaHuaY in #3407
• GH-3215: Sort the list of org.apache.parquet.format.Encoding created … by @mbrannstrom in #3413
• Bump dependabot limit by @Fokko in #3416
• Bump org.mockito:mockito-core from 5.21.0 to 5.22.0 by @dependabot[bot] in #3436
• Bump org.apache:apache from 33 to 37 by @dependabot[bot] in #3431
• Bump protobuf.version from 4.33.2 to 4.34.0 by @dependabot[bot] in #3441
• Bump com.github.luben:zstd-jni from 1.5.7-3 to 1.5.7-7 by @dependabot[bot] in #3442
• Bump org.apache.maven.plugins:maven-shade-plugin from 3.6.0 to 3.6.2 by @dependabot[bot] in #3430
• Bump org.xerial.snappy:snappy-java from 1.1.10.7 to 1.1.10.8 by @dependabot[bot] in #3426
• Bump org.codehaus.mojo:exec-maven-plugin from 3.5.0 to 3.6.3 by @dependabot[bot] in #3440
• Bump org.apache.parquet:parquet-pig from 1.15.0 to 1.15.2 by @dependabot[bot] in #3420
• Bump commons-logging:commons-logging from 1.3.5 to 1.3.6 by @dependabot[bot] in #3418
• Bump com.google.re2j:re2j from 1.7 to 1.8 by @dependabot[bot] in #3433
• Bump org.apache.commons:commons-lang3 from 3.17.0 to 3.20.0 by @dependabot[bot] in #3429
• GH-3411 Expose row group index via Parquet reader by @uros7251brick in #3412
• Bump org.mockito:mockito-core from 5.22.0 to 5.23.0 by @dependabot[bot] in #3445
• Bump net.openhft:zero-allocation-hashing from 0.27ea0 to 0.27ea1 by @dependabot[bot] in #3419
• Bump org.tukaani:xz from 1.10 to 1.12 by @dependabot[bot] in #3417
• Bump it.unimi.dsi:fastutil from 8.5.15 to 8.5.18 by @dependabot[bot] in #3422
• Bump org.apache.commons:commons-text from 1.13.1 to 1.15.0 by @dependabot[bot] in #3438
• GH-3449: Fix BinaryStats int overflow by @Jiayi-Wang-db in #3448
• Bump protobuf.version from 4.34.0 to 4.34.1 by @dependabot[bot] in #3453
• Bump org.apache.arrow:arrow-vector from 17.0.0 to 19.0.0 by @dependabot[bot] in #3454
• GH-3410: Bump jackson dependencies from 2.19.2 to 2.21.2 by @manuzhang in #3460
• Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.23.1 to 0.25.4 by @dependabot[bot] in #3437
• Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.4 to 0.25.5 by @dependabot[bot] in #3473
• Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.5 to 0.25.6 by @dependabot[bot] in #3491
• Bump com.google.guava:guava from 33.5.0-jre to 33.6.0-jre by @dependabot[bot] in #3492
• GH-3011: Deny further writes after InternalParquetRecordWriter is aborted by @LuciferYang in #3450
• GH-3461: Avoid unnecessary debug logging overhead in record write hot path by @arouel in #3462
• [GH-3408] Add support for stale PR workflow by @ArnavBalyan in #3409
• Remove Parquet-Pig by @dossett in #3377
• Bump com.google.api.grpc:proto-google-common-protos from 2.59.2 to 2.68.0 by @dependabot[bot] in #3474
• Bump org.apache.maven.plugins:maven-dependency-plugin from 3.8.1 to 3.10.0 by @dependabot[bot] in #3425
• PARQUET-3532: Remove hardcoded versions from pom.xml by @Fokko in #3531
• Bump org.codehaus.mojo:build-helper-maven-plugin from 3.6.0 to 3.6.1 by @dependabot[bot] in #3434
• GH-3457: Validate column index for null_pages/null_counts contradiction by @clee704 in #3458
• GH-3112: Fix reading of proto Uint32Value by @0x26res in #3113
• Bump jackson.version from 2.21.2 to 2.21.3 by @dependabot[bot] in #3538
• Bump commons-io:commons-io from 2.21.0 to 2.22.0 by @dependabot[bot] in #3528
• Bump com.github.luben:zstd-jni from 1.5.7-7 to 1.5.7-8 by @dependabot[bot] in #3541
• GH-3464: Improve DeltaByteArrayWriter.writeBytes to avoid unnecessary allocation and scalar prefix comparison by @arouel in #3465
• GH-3478: Fix LZ4_RAW heap decompression by @yadavay-amzn in #3486
• GH-3497: Fix thread-unsafe singleton in DefaultValuesWriterFactory by @arouel in #3498
• GH-3261: Fix integer overflow in CapacityByteArrayOutputStream by @yadavay-amzn in #3525
• GH-3545: Use parquet-java in SCM block by @RussellSpitzer in #3546
• MINOR: Fix Spotless formatting in TestCompressionCodec by @RussellSpitzer in #3551
• Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.6 to 0.25.7 by @dependabot[bot] in #3554
• Bump com.google.api.grpc:proto-google-common-protos from 2.68.0 to 2.71.0 by @dependabot[bot] in #3553
• GH-3451. Add a JMH benchmark for variants by @steveloughran in #3452
• GH-3414: Add parseJson to VariantBuilder for JSON-to-Variant conversion by @gaurav7261 in #3415
• Bump actions/stale from 9 to 10 by @dependabot[bot] in #3527
• GH-3482: parquet-hadoop tests to work behind a web proxy by @alexeyroytman in #3483
• GH-3471: Fix ByteBuffer handling in VariantUtil and VariantBuilder by @rayokota in #3472
• GH-3484: Eliminate per-page heap allocation for CRC32 checksums when using direct ByteBufferAllocator by @arouel in #3485
• Optimizing Variant read path with lazy caching by @nssalian in #3481
• Bump com.github.luben:zstd-jni from 1.5.7-8 to 1.5.7-9 by @dependabot[bot] in #3581
• Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.7 to 0.26.0 by @dependabot[bot] in #3583
• GH-3573: Fix VariantUtil string decoding to use explicit UTF-8 charset by @mikhail-melnik in #3576
• Bump protobuf.version from 4.34.1 to 4.35.0 by @dependabot[bot] in #3582
• GH-3587: Fix parquet.thrift.string.size.limit validation by @divjotarora in #3588
• GH-3572: Bump thrift to 0.23 by @steveloughran in #3589
• Bump org.apache:apache from 37 to 38 by @dependabot[bot] in #3564
• Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.26.0 to 0.26.1 by @dependabot[bot] in #3592
• Bump com.fasterxml.jackson.core:jackson-databind from 2.21.3 to 2.21.4 by @dependabot[bot] in #3593
• Bump jackson.version from 2.21.3 to 2.21.4 by @dependabot[bot] in #3594
• GH-3398: Fix potential ClassLoader leak caused by ThreadLocal lambda in Binary.java by @LuciferYang in #3447
• Bump parquet-format to 2.13.0 by @wgtmac in #3611
• Bump protobuf.version from 4.35.0 to 4.35.1 by @dependabot[bot] in #3612
• Bump com.github.luben:zstd-jni from 1.5.7-9 to 1.5.7-11 by @dependabot[bot] in #3613
• Bump com.google.api.grpc:proto-google-common-protos from 2.71.0 to 2.72.0 by @dependabot[bot] in #3602
• GH-3574: parquet-hadoop: Statistics.toParquetStatistics: always set null_count by @mdibaiee in #3575
• Bump com.fasterxml.jackson.core:jackson-databind from 2.21.4 to 2.22.0 by @dependabot[bot] in #3606
• MINOR: Remove Java 8 direct buffer cleaner path by @dossett in #3591
• Simplify CI: collapse codec matrix and add zstd coverage by @iemejia in #3580
• Bump jackson.version from 2.21.4 to 2.22.0 by @dependabot[bot] in #3605
• Bump com.fasterxml.jackson.core:jackson-annotations from 2.21 to 2.22 by @dependabot[bot] in #3604
• GH-3489: Fix Binary.copy() to materialize direct ByteBuffer-backed values to heap by @arouel in #3490
• GH-3632: Fix record-level notIn evaluation by @wgtmac in #3631
• Bump commons-logging:commons-logging from 1.3.6 to 1.4.0 by @dependabot[bot] in #3623
• GH-3627: Add missing onAppend() to VariantBuilder#appendUUIDBytes by @nastra in #3624
• Bump com.squareup.okhttp3:okhttp from 4.12.0 to 5.4.0 by @nastra in #3630
• GH-3598: Expose getRowRanges(int) by @peter-toth in #3599
• GH-3585: Fix DataPageHeaderV2.num_nulls=-1 when column statistics are disabled by @keuin in #3586
• PARQUET-2249: Add IEEE-754 total order and nan count for floating types by @wgtmac in #3393
• GH-3618: Upgrade JUnit4 to JUnit 5.14.4 by @nastra in #3619
• GH-3542 Fix LocalInputFile ByteBuffer read methods to support direct and non-zero-position buffers by @arouel in #3543
• Bump org.cyclonedx:cyclonedx-maven-plugin from 2.9.1 to 2.9.2 by @dependabot[bot] in #3635
• GH-3628: prevent NPE & unclosed releaser by @nastra in #3629
• GH-3625: Compute dataSize correctly when dealing with duplicate keys by @nastra in #3626
• GH-3615: Introduce AssertJ for testing by @nastra in #3616
• Restrict trigger push branch for GitHub Workflow by @apupier in #3641
• GH-3639: Don't drop row groups for IN(..., null) when num_nulls is not set by @nastra in #3640
• GH-3615: Use assertThat checks in parquet-variant tests by @nastra in #3617
• GH-3615: Replace @test(expected=...) with assertThatThrownBy() checks by @nastra in #3636
• Bump org.apache:apache from 38 to 39 by @dependabot[bot] in #3646
• GH-3475: Fix parquet-vector compatiblity with Java > 17 by @iemejia in #3476
• GH-3621: Replace the previous field in the queue instead of appending to the queue by @njlaw in #3622
• MINOR: Fix PrimitiveStringifier class Javadoc to link UnsupportedOperationException by @anxkhn in #3651
• GH-3530: Optimize PLAIN encoding and decoding with direct ByteBuffer I/O by @iemejia in #3565
• Bump com.google.api.grpc:proto-google-common-protos from 2.72.0 to 2.73.0 by @dependabot[bot] in #3657
• Bump jackson.version from 2.22.0 to 2.22.1 by @dependabot[bot] in #3656
• Bump com.fasterxml.jackson.core:jackson-databind from 2.22.0 to 2.22.1 by @dependabot[bot] in #3658
• GH-3596: Add RowRanges.Builder for incremental construction from selected row indices by @peter-toth in #3597
• PARQUET-3459: Per column compression by @mengna-lin in #3526
• Add SECURITY.md by @emkornfield in #3643
• GH-3659: Throw IAE when dealing with odd-length/invalid hex string in Parquet CLI by @nastra in #3642
• GH-3637: Fix corrupt output when reusing a dictionary writer across row groups after fallback by @nastra in #3638
• GH-3644: Handle NoClassDefFoundError in DynMethods by @nastra in #3645
• MINOR: Fix withLogicalTypeAnnotation Javadoc @return description by @anxkhn in #3648
• MINOR: Fix VariantUtil.getByte() Javadoc copied from getShort() by @anxkhn in #3649
• GH-3652: Use assertThat checks in parquet-plugins/parquet-protobuf tests by @nastra in #3653
• Fix CI caused by a silent merge conflict by @nastra in #3665
• [INFRA] Set up default rulesets for default and release branches by @asf-gitbox-commits in #3563
• GH-3652: Use assertThat checks in parquet-common tests by @nastra in #3654
• GH-3652: Use assertThat checks in parquet-avro tests by @nastra in #3660
• GH-3652: Use assertThat checks in parquet-cli tests by @nastra in #3661
• GH-3615: Use assertThat checks in parquet-column tests by @nastra in #3664
• GH-3652: Use assertThat checks in parquet-format-structures/parquet-encoding/parquet-thrift tests by @nastra in #3650
• GH-3671: Keep files readable when they use future logical types by @kevinjqliu in #3669
• GH-3652: Use assertThat checks in parquet-hadoop tests by @nastra in #3666
• GH-3618: Migrate multiple modules to JUnit5 by @nastra in #3673
• Fix CI due to silent merge conflict with AssertJ migration by @nastra in #3677

New Contributors

• @takezoe made their first contribution in #3385
• @HuaHuaY made their first contribution in #3400
• @mbrannstrom made their first contribution in #3413
• @uros7251brick made their first contribution in #3412
• @manuzhang made their first contribution in #3460
• @arouel made their first contribution in #3462
• @clee704 made their first contribution in #3458
• @0x26res made their first contribution in #3113
• @yadavay-amzn made their first contribution in #3486
• @RussellSpitzer made their first contribution in #3546
• @gaurav7261 made their first contribution in #3415
• @alexeyroytman made their first contribution in #3483
• @rayokota made their first contribution in #3472
• @nssalian made their first contribution in #3481
• @mikhail-melnik made their first contribution in #3576
• @divjotarora made their first contribution in #3588
• @mdibaiee made their first contribution in #3575
• @peter-toth made their first contribution in #3599
• @keuin made their first contribution in #3586
• @apupier made their first contribution in #3641
• @njlaw made their first contribution in #3622
• @anxkhn made their first contribution in #3651
• @mengna-lin made their first contribution in #3526
• @asf-gitbox-commits made their first contribution in #3563
• @kevinjqliu made their first contribution in #3669

Full Changelog: apache-parquet-1.17.0...apache-parquet-1.18.0-rc1