pyiceberg-0.12.0

What's Changed

• Catalog tests around nonexistent tables + namespaces by @rambleraptor in #2990
• Upgrade to RAT 0.17 by @kevinjqliu in #2992
• Fix: make test-coverage to properly enable coverage for all test suites by @kevinjqliu in #2989
• feat: Add tell() to OutputStream writers by @geruh in #2998
• infra: upgrade pytest and use pytest_lazy_fixtures by @kevinjqliu in #2978
• fix: ignore local config during tests by @geruh in #3006
• fix: Hive catalog test failure on CPython 3.13.12 by @geruh in #3043
• docs: release docs improvements by @kevinjqliu in #3030
• Add catalog properties for Catalog Tests by @rambleraptor in #2982
• feat: Add Geometry & Geography Types by @SaymV in #2859
• infra: set github actions max-parallel to 15 by @kevinjqliu in #3057
• fix: Accept all valid http method for IRC's ConfigResponse by @sumanth-manchala in #3010
• feat(cli): add --prefix option for REST catalog by @mrutunjay-kinagi in #3061
• CI: Add CodeQL workflow for GitHub Actions security scanning by @kevinjqliu in #3060
• Surface too many requests exception by @ms32035 in #3048
• fix: add Field alias for SetPartitionStatisticsUpdate.partition_stati… by @shiwk in #3035
• fix ssl deactivation from catalog properties #2985 by @MenelikV in #3012
• Optimization: Prune manifest in snapshot overwrite operations by @gabeiglio in #3011
• Add concurrency safety validation checks by @gabeiglio in #3049
• infra: simplify Makefile and uv by @kevinjqliu in #2996
• fix: Extract ADLS account_name from URI hostname in FsspecFileIO by @antonlin1 in #3005
• Fix conftest dependency breaking nightly wheel build by @kevinjqliu in #3078
• chore: Remove docs from uv default groups by @geruh in #3073
• fix: show helpful error when hive catalog config is missing uri by @jx2lee in #3077
• chore(ci): add explicit least-privilege workflow permissions by @kevinjqliu in #3082
• fix: python version in release docs workflow by @geruh in #3102
• chore: use ubuntu-slim for lightweight jobs by @zhjwpku in #3107
• Fix pyparsing diagnostic warning on Table import by @kevinjqliu in #3081
• Documentation Adjustment for s3.profile-name and client.profile-name by @thomas-pfeiffer in #3085
• docs: more release docs improvements by @kevinjqliu in #3104
• feat(cli): replace REST --prefix with --warehouse by @kevinjqliu in #3080
• Update slate config for PRs by @nssalian in #3125
• Support storage-credentials in REST catalog LoadTableResult by @rcjverhoef in #3042
• feat(rest): add sigv4 retry configuration defaults by @mrutunjay-kinagi in #3063
• [View Support] Add ViewMetadata read support and create_view to REST Catalog by @rambleraptor in #2154
• fix: handle empty HEAD response by @AntonioYuen in #3041
• Switch to uv-pre-commit for lockfile management by @kevinjqliu in #3141
• Improve lockfile management documentation and enforce uv.lock freshness in CI by @kevinjqliu in #3144
• Fix for ns timestamp non utc by @nssalian in #3142
• Skip catalog loading for the version command by @jx2lee in #3146
• docs: add type mapping tables between PyIceberg and PyArrow by @iamluan in #3098
• fix(glue): Support create_table for S3 Tables federated databases by @jamesbornholt in #3058
• ci: pin third-party actions to Apache-approved SHAs by @kevinjqliu in #3172
• Infra: Remove GitHub Actions from Dependabot configuration by @kevinjqliu in #3187
• Infra: Pin pypa/cibuildwheel version by @ebyhr in #3189
• chore(ci): pin actions in PR-triggered workflows by @kevinjqliu in #3194
• chore(ci): pin actions in non-PR-triggered workflows by @kevinjqliu in #3193
• ci: improve github workflows by @kevinjqliu in #3204
• ci: add tag as inline comment by @kevinjqliu in #3213
• ci: fix nightly and release by @kevinjqliu in #3215
• fix: Handle optional properties in load namespace properties response by @geruh in #3169
• CI: Lock REST Fixture to 1.10.1 by @Fokko in #3240
• fix zizmor ci issue by bumping codeql-action to v4.35.2 by @geruh in #3245
• ci: fix zizmor workflow by @kevinjqliu in #3243
• Fix ManifestEntry.snapshot_id setter writing to wrong index by @lawofcycles in #3257
• Fix pa.Schema type annotations in schema_to_pyarrow by @davidbzhao in #3261
• perf: build partition filter with balanced tree to avoid RecursionError by @yingjianwu98 in #3264
• bug: NoopCatalog should return False, not throw error by @rambleraptor in #3284
• Reading upper/lower bounds values with type promotions by @rambleraptor in #3293
• Fix DELETED manifest entry snapshot_id in OverwriteFiles by @lawofcycles in #3237
• infra: add python 3.14 support by @afeldman1 in #3299
• fix: SaslServer busy-loop causes process hang after test suite by @kevinjqliu in #3305
• perf: Hoist table_metadata at remaining repeat-access sites in snapshot update by @rynewang in #3301
• Fix deepcopy for And, Or, and Not expressions by @robreeves in #3295
• Add support for 'overwrite' option in register_table by @ebyhr in #3290
• Document TableAlreadyExistsError in catalog.rename_table by @ebyhr in #3289
• Initial work for file format writer API by @nssalian in #3119
• fix:update config during table refresh by @yingjianwu98 in #3327
• Remove unused code by @ebyhr in #3326
• Update NOTICE year 2025 -> 2026 by @kevinjqliu in #3332
• CI: Use specific patch versions in workflow action comments by @kevinjqliu in #3331
• feat: add load_view to REST catalog by @spr0els in #3224
• Add support for registering views by @ebyhr in #3288
• fix(tests): isolate state in test_write_optional_list by @paultmathew in #3334
• add papermill-based notebook tests for pyiceberg examples by @federicsp in #3330
• REST: Add pagination support for list_views by @ebyhr in #3349
• fix(fsspec): handle zero-byte files in len by @fallintoplace in #3353
• Add @typing.override annotations to catalog and FileIO implementations by @vishnuprakaz in #3360
• Fix typo in test_add_files by @ebyhr in #3378
• fix: avoid TSaslClientTransport reuse to eliminate server-side SASL noise by @barking-code in #3357
• feat(2152): support pa.RecordBatchReader in Table.append/overwrite by @paultmathew in #3335
• REST: Rename rest-scan-planning-enabled to scan-planning-mode by @ebyhr in #3376
• fix(table): avoid committing update builders after exceptions by @fallintoplace in #3354
• perf(add_files): stream manifest entries for duplicate-files check by @laichunpongben in #3287
• Build: Update actions/upload-artifact to v7.0.1 by @ebyhr in #3401
• docs: fix New York longitude sign in partial overwrite example by @fallintoplace in #3403
• Test: Replace rest-scan-planning-enabled with scan-planning-mode by @ebyhr in #3391
• feat(view): View object API by @gabeiglio in #3338
• REST: Add pagination support for list_namespaces by @ebyhr in #3347
• REST: Add pagination support for list_tables by @ebyhr in #3348
• Make View.sql_for case-insensitive and return None for unknown dialect by @ebyhr in #3407
• REST: Add integration test for views by @ebyhr in #3406
• REST: Add support for page-size in list_namespaces, list_tables, and list_views by @ebyhr in #3377
• REST: Allow Identifier type in drop_view method by @ebyhr in #3408
• feat(cli): deprecate version command in favor of --version flag by @chrisqiqiu in #3206
• Fix walrus truthiness on metrics bounds and identity-partition projection by @kevinjqliu in #3412
• Reject empty source-ids in PartitionField / SortField by @kevinjqliu in #3411
• Fix precision loss in large integral string conversions by @fallintoplace in #3405
• Chore: Updated PyArrow Dependencies by @CalebWeisgerber in #3362
• Fix typo by @ebyhr in #3424
• docs: add Iceberg security model by @sungwy in #3425
• Add rambleraptor to collaborators list by @kevinjqliu in #3456
• Make View.version return None for unknown id by @ebyhr in #3445
• DOCS: Fix layout of sqlite warning by @ebyhr in #3459
• Fix misleading file extension for partition statistics by @ebyhr in #3472
• fix(literals): return long bounds for decimal conversion by @abnobdoss in #3470
• Add defaults to ViewVersion fields by @ebyhr in #3458
• Update astral-sh/setup-uv to v8.2.0 by @ebyhr in #3510
• feat: add dictionary_columns to to_arrow() / to_arrow_batch_reader() for memory-efficient reads by @GayathriSrividya in #3461
• Refactor: extract BaseScan and ManifestGroupPlanner by @smaheshwar-pltr in #3511
• Add more examples for pyiceberg view by @MonkeyCanCode in #3414
• feat: Add REST loadCredentials support by @kevinjqliu in #3499
• fix: correct NOT STARTS WITH projection for truncated partitions by @GayathriSrividya in #3528
• Fix NotStartsWith residual evaluation to return correct result by @ebyhr in #3503
• Return overflow sentinel in LongLiteral.to(FloatType) by @ebyhr in #3502
• Fix string-based starts_with and not_starts_with methods by @ebyhr in #3501
• Return bounds sentinels for long date literals by @kevinjqliu in #3546
• Fix duplicate filtering path in Arrow task batches by @GayathriSrividya in #3448
• docs: clean OSS and COS configuration tables by @Sidra-009 in #3452
• Fix strict NotEqualTo/NotIn pruning with partial nulls or NaNs by @tanmayrauth in #3521
• Add strict NotEqualTo/NotIn null and NaN tests by @kevinjqliu in #3547
• CI: Check ASF action allowlist on every PR by @ebyhr in #3550
• fix(datetime): raise specific "Missing zone offset" error in timestamptz_to_nanos by @vishnuprakaz in #3505
• Extract DeletionVector logic from PuffinFile by @ebyhr in #3491
• Feature: Incremental Append Scan by @smaheshwar-pltr in #3512
• Make scan_plan_helper internal (#3541) by @tanmayrauth in #3553
• Make manifest cache size configurable by @kris-gaudel in #2993
• Rename _scan_plan_helper method to _plan_manifest_entries by @ebyhr in #3579
• Infra: Set timeout for jobs in python-ci.yml by @ebyhr in #3581
• Reject unsupported identity transform types by @zhjwpku in #3517
• Prune dependency upper bounds by @kevinjqliu in #3586
• CI: Pass args through docker-compose by @Fokko in #3587
• Fix strip spec-mandated DV blob framing before deserializing by @ebyhr in #3576
• Docs: Add missing _downcast_ns_timestamp_to_us to ArrowScan docstring by @qzyu999 in #3594
• fix: require pyarrow>=18.0.0 for native UUID type support by @anxkhn in #3606
• Build: Add .venv/ to .gitignore by @qzyu999 in #3626
• fix(ci): correct stale-issue-label and exempt security issues by @anxkhn in #3605
• fix: preserve manifest min sequence number of 0 by @anxkhn in #3660
• fix: raise ValidationError for decimal precision outside 1-38 range by @PLTNGM in #3585
• Support zstd blob decompression in Puffin by @ebyhr in #3575
• Add basic assumptions to AGENTS.md by @rambleraptor in #3552
• Infra: Group github/codeql-action bumps into a single dependabot PR by @ebyhr in #3648
• feat: support pyarrow float16 by widening to float on read/write by @anxkhn in #3590
• fix: Pass metadata location to StaticTable FileIO for scheme inference by @kdn36 in #3592
• fix: preserve write_default when applying a name mapping by @anxkhn in #3601
• Only run integration tests on Python changes by @rambleraptor in #3610
• Implement ParquetFormatModel and update write_file to use the format API by @nssalian in #3381
• fix: preserve dictionary encoding in to_arrow_batch_reader by @avison9 in #3595
• fix: add iceberg_type column for SqlCatalog by @rchowell in #3263
• docs: fix undefined variable in schema evolution transaction example by @anxkhn in #3607
• Make partition expression evaluation stateless by @dossett in #3664
• Convert FileFormatModel from ABC to typing.Protocol by @nssalian in #3692
• chore(ci): Update ASF allowlist check action dependency to track versioned releases by @kevinjqliu in #3739
• build: bump pyiceberg-core from 0.9.1 to 0.10.1 by @VVKot in #3711
• Remove stale VariantType transform TODO by @zhjwpku in #3742
• ci: enable ASF Copilot code review by @kevinjqliu in #3740
• fix: preserve empty property values in CLI by @mj006648 in #3745
• CI: Add Windows unit test job by @qzyu999 in #3723
• refactor: simplify CLI property lookup by @mj006648 in #3750
• Update release docs for uv lock by @rambleraptor in #3749
• INFRA: Use PR titles for squash commits by @kevinjqliu in #3755
• Fix flaky moto_server fixture by using an OS-assigned ephemeral port by @puchengy in #3759
• Docs: Update Iceberg Slack link by @ebyhr in #3768
• Add --purge option to drop table CLI by @jx2lee in #3718
• Release verification script by @rambleraptor in #3777
• Fix/sqlcatalog iceberg type filter by @avison9 in #3709
• Make metrics evaluation stateless by @dossett in #3665
• Add commit retry and concurrency validation for writes by @lawofcycles in #3320
• Add docstrings to pyiceberg/table/inspect.py by @daviddallakyan2005 in #3802
• feat: add S3 SSE configs (FsspecFileIO only) by @achasnovskiy in #3173
• Add Spark interop test for reading Puffin deletion vectors by @moomindani in #3476
• fix: correct class name in Long{AboveMax,BelowMin} type-change error by @anxkhn in #3634
• fix(fsspec): parse S3 virtual addressing as a boolean by @fallintoplace in #3492
• fix: use removeprefix to decode DynamoDB namespace property keys by @anxkhn in #3602
• fix: raise TypeError for unsupported partition source type by @anxkhn in #3617
• fix: compare timestamps for partitions metadata last_updated_at by @anxkhn in #3603
• test: fix operator precedence in manifest v2 assertions by @anxkhn in #3612
• fix: pad sub-microsecond digits when parsing nanosecond timestamps by @anxkhn in #3614
• fix: stop calling deprecated PuffinFile.to_vector in Spark DV test by @daviddallakyan2005 in #3804
• bug: fix delete_data_file on partitioned tables by @gabeiglio in #3780
• Validate replaced data files during commit retries by @kevinjqliu in #3811
• Infra: Enable pull request auto-merge by @kevinjqliu in #3815
• fix(decimal): use minimal byte length for negative powers of two by @vishnuprakaz in #3746
• Fix residual NotNaN for null partition values by @BharatDeva in #3689
• Fix strict evaluation for negative record counts by @kevinjqliu in #3831
• Fail when explicitly deleted data file is missing by @rambleraptor in #3818

New Contributors

• @SaymV made their first contribution in #2859
• @mrutunjay-kinagi made their first contribution in #3061
• @shiwk made their first contribution in #3035
• @MenelikV made their first contribution in #3012
• @antonlin1 made their first contribution in #3005
• @jx2lee made their first contribution in #3077
• @zhjwpku made their first contribution in #3107
• @thomas-pfeiffer made their first contribution in #3085
• @rcjverhoef made their first contribution in #3042
• @AntonioYuen made their first contribution in #3041
• @iamluan made their first contribution in #3098
• @jamesbornholt made their first contribution in #3058
• @lawofcycles made their first contribution in #3257
• @davidbzhao made their first contribution in #3261
• @afeldman1 made their first contribution in #3299
• @rynewang made their first contribution in #3301
• @robreeves made their first contribution in #3295
• @spr0els made their first contribution in #3224
• @paultmathew made their first contribution in #3334
• @federicsp made their first contribution in #3330
• @barking-code made their first contribution in #3357
• @laichunpongben made their first contribution in #3287
• @chrisqiqiu made their first contribution in #3206
• @CalebWeisgerber made their first contribution in #3362
• @abnobdoss made their first contribution in #3470
• @GayathriSrividya made their first contribution in #3461
• @MonkeyCanCode made their first contribution in #3414
• @Sidra-009 made their first contribution in #3452
• @tanmayrauth made their first contribution in #3521
• @qzyu999 made their first contribution in #3594
• @PLTNGM made their first contribution in #3585
• @kdn36 made their first contribution in #3592
• @avison9 made their first contribution in #3595
• @dossett made their first contribution in #3664
• @VVKot made their first contribution in #3711
• @mj006648 made their first contribution in #3745
• @daviddallakyan2005 made their first contribution in #3802
• @achasnovskiy made their first contribution in #3173
• @moomindani made their first contribution in #3476
• @BharatDeva made their first contribution in #3689

Full Changelog: pyiceberg-0.11.1...pyiceberg-0.12.0