Skip to content

Releases: delta-io/delta-rs

python-v0.6.3

09 Nov 17:33
760b30a
Compare
Choose a tag to compare

What's Changed

New Contributors

Full Changelog: python-v0.6.2...python-v0.6.3

What's Changed

New Contributors

Full Changelog: python-v0.6.2...python-v0.6.3

python-v0.6.2

11 Oct 06:14
5ce4a40
Compare
Choose a tag to compare

What's Changed

Full Changelog: python-v0.6.1...python-v0.6.2

python-v0.6.1

06 Sep 15:55
45a0404
Compare
Choose a tag to compare

What's Changed

  • feat: add gcs integration tests by @roeap in #779
  • build(deps): bump lz4-sys from 1.9.2 to 1.9.4 in /aws/delta-checkpoint by @dependabot in #782
  • build(deps): bump lz4-sys from 1.9.2 to 1.9.4 in /delta-inspect by @dependabot in #783
  • build(deps): bump tokio from 1.20.1 to 1.21.0 by @dependabot in #790
  • build(deps): bump thiserror from 1.0.32 to 1.0.34 by @dependabot in #792
  • build(deps): bump pretty_assertions from 1.2.1 to 1.3.0 by @dependabot in #791
  • build(deps): bump anyhow from 1.0.62 to 1.0.64 by @dependabot in #793
  • build(deps): bump env_logger from 0.7.1 to 0.9.0 by @dependabot in #794
  • hotfix: python object store paths by @roeap in #787
  • prepare python release 0.6.1 by @roeap in #795

Full Changelog: python-v0.6.0...python-v0.6.1

python-v0.6.0

31 Aug 16:30
174cf84
Compare
Choose a tag to compare

What's Changed

New Contributors

Full Changelog: python-v0.5.8...python-v0.6.0

python-v0.5.8

23 Jun 19:46
27c0cbb
Compare
Choose a tag to compare

What's Changed

  • Expose read and write options in public API by @george-zubrienko in #581
  • [proof] make sure lock at least expires once by @houqp in #591
  • Python API - delta.appendOnly enforcement by @WarSame in #590
  • Avoid building pandas and numpy from source by @wjones127 in #595
  • Introduce require_files for tracking the add files in table state by @mosyp in #594
  • Make sure pandas is optional by @wjones127 in #597
  • High level Delta Operations with Datafusion by @roeap in #584
  • Re-enable datafusion tests and improve supported types. by @roeap in #601
  • default to root for empty path in azure store by @roeap in #603
  • publish dynamodb_lock to crates.io by @houqp in #605
  • Configure Azure storage using a map (#555) by @Blajda in #598
  • Azure options by @roeap in #606
  • Update rusoto dependencies to 0.48 by @ahmedriza in #611
  • upgrade to datafusion 8 by @houqp in #612
  • fix: cap sphinx version to avoid bug in 5.0 by @wjones127 in #615
  • Provide Python aarch64 wheels for Linux. by @fvaleye in #613
  • Refactoring of the Python release Github action by @fvaleye in #616
  • fix: Use relative paths for add paths by @wjones127 in #618
  • Bin packing optimization by @Blajda in #607
  • feat: impl rename_noreplace with std::fs::hard_link by default by @wjones127 in #621
  • feat(python): validate schema in write_deltalake by @wjones127 in #624
  • Fix the AWS_REGION environment variable configuration in S3 backend by @fvaleye in #633
  • Refactor azure storage with crate updates by @roeap in #644
  • Defer creation of storage backend in DeltaTableBuilder by @Blajda in #639
  • fix: Add correct size and null paritition values to add actions by @wjones127 in #625
  • Bump flatbuffers from 0.8.4 to 2.1.2 in /aws/delta-checkpoint by @dependabot in #626
  • Bump hyper from 0.14.9 to 0.14.19 in /aws/delta-checkpoint by @dependabot in #628
  • Bump regex from 1.5.4 to 1.5.5 in /aws/delta-checkpoint by @dependabot in #629
  • Bump regex from 1.5.4 to 1.5.6 in /delta-inspect by @dependabot in #630
  • Bump thread_local from 1.1.3 to 1.1.4 in /aws/delta-checkpoint by @dependabot in #646
  • fix: Prevent warning spam when reading tables generated by delta 1.2.1 by @Tom-Newton in #651
  • refactor: move version field to DeltaTableState by @roeap in #649
  • feat: add enforce_retention_duration param to vacuum method by @houqp in #648
  • fix: read vacuumed delta log without _last_checkpoint by @roeap in #643
  • feat: Upgrade to arrow/parquet 15 and datafusion 9 by @xianwill in #652
  • Release of the Python binding version 0.5.8 by @fvaleye in #640

New Contributors

Full Changelog: python-v0.5.7...python-v0.5.8

python-v0.5.7

20 Apr 07:27
4aad6aa
Compare
Choose a tag to compare

What's Changed

New Contributors

Full Changelog: python-v0.5.6...python-v0.5.7

python-v0.5.6

06 Feb 20:41
2fa81c6
Compare
Choose a tag to compare
  • Bump version of Python binding to 0.5.6 (#558)
  • Move delta-inspect to its own crate (#557)
  • Fix VACUUM by using table_uri when filtering files to delete (#551)
  • Formally verify S3 atomic rename (#540)
  • Implement missing Azure storage backend methods (#499)
  • Implement polling for table updates (#550)
  • Add target in Python release Github action workflow. (#548)

Credits:
QP Hou, Thomas Vollmer, David Blajda, Florian Valeye

Full Changelog: python-v0.5.5...python-v0.5.6

python-v0.5.5

24 Jan 17:42
8fb45b2
Compare
Choose a tag to compare
  • Add storage options for backends (#544)
  • Remove coupling of DynamoDbLockClient from S3 storage (#535)
  • add macOS 11 support in python binding release (#541)
  • Refresh Python usage documentation (#539)
  • [Python] Create PyArrow dataset fragments from delta log (#525)
  • Fix Delta metadata transaction schema (#531)
  • Add gcs test and improve credential error (#533)
  • Return complete history (#526)
  • Move dynamodb lock into its own crate (#508)
  • Add datafusion examples to docs (#519)
  • Fix S3 list_objs and cleanup_metadata (#518)
  • Add support for creating List and Map schema types (#517)
  • Update datafusion version to 6 (#516)
  • Retry S3 get request on 500 Internal Server Error (#510)
  • Fix memory overhead when creating checkpoint (#502)
  • Fix nullable partition values (#498)
  • Fix cleanup_expired_logs timestamp (#503)
  • Add bool config enableExpiredLogCleanup. (#500)
  • pin arrow to major version (#501)

Credits:
Florian Valeye, ahmedriza, Will Jones, Liang-Chi Hsieh, Gabriel J. Michael, Matthew Turner, Mykhailo Osypov, Andrei Ionescu, QP Hou

Full Changelog: python-v0.5.4...python-v0.5.5

python-v0.5.4

17 Nov 08:35
8a0475c
Compare
Choose a tag to compare
  • Clean up expired delta table commit logs after checkpoint (#484)
  • Add authorization options for azure storage backend (#486)
  • Bump arrow to 6.1.0 (#494)
  • Add DeltaTableError in Python binding. Add markers for integration tests with pytest. (#496)
  • Change Rust edition from 2018 to 2021 (#490)
  • Add docs for ADLS Gen2. (#492)
  • Add gt, gte, lt and lte partition filters. (#478)
  • Fix python build (#487)
  • Try to fix flaky rename under Windows (#485)
  • Update azure crates (#474)
  • Update README.adoc (#482)
  • Fix documentation for the DeltaStorageHandler (#483)
  • Throw an error when filter key is not in partitioned columns. (#475)
  • Add GCS feature to the Python Cargo.toml file (#476)
  • Make file storage backend's atomic rename async (#471)
  • materialize tables in python via native storage backend (#463)
  • Fix coverage of the Python tests (#467)
  • Support hash lookup by path string for Remove action (#462)
  • Add new module for DeltaTableState (#464)
  • Avoid table stats override in datafusion extension. (#459)
  • Fix action reconciliation for add after remove (#456)
  • Add pool_idle_timeout options for s3 and sts clients (#458)
  • Generate new session name on assume role credentials provider refresh (#451)
  • return lazy iterator in get tombstone methods (#452)
  • Support no tombstone loading & new table builder API (#445)
  • Fix broken tombstones metadata when extended_file_metadata is different between tomstones in state (#450)
  • README: mark Checkpoint creation as done for Rust (#449)
  • Add maturin develop command with extra (#448)
  • Run all tests under s3 feature flag (#447)
  • Update datafusion links (#446)
  • Batch-apply remove actions in tombstone handling (#444)
  • Fixing test to compare sorted vec (#443)
  • Add delete_lock and fix release_lock (#440)

Credits:
Liang-Chi Hsieh, Robert Pack, Mykhailo Osypov, Florian Valeye, Thomas Vollmer, Yuan Zhou, roeap, Denny Lee, Yuan Zhou, Kelvin S. do Prado, QP Hou, Thomas Peiselt, Bruno Bigras, Akshay Ghiya

python-v0.5.3

21 Sep 10:14
40d3d90
Compare
Choose a tag to compare
  • Add history command in delta-rs (#428)
  • reenable datafusion integration with temporary fork (#436)
  • Decode path in Add and Remove actions. (#434)
  • Optimize remove action apply with early iteration exit #424 (#431)
  • Clean up DeltaTransactionError (#432)
  • Add is_non_acquirable field to the dynamodb lock (#429)
  • Expose valid primitive type list to public doc (#430)
  • Support partition value string deserialization for timestamp/binary (#371)
  • Bump arrow to 6.0.0-SNAPSHOT and bring map support to schema (#375)
  • Update README.adoc (#426)
  • Introduce DeltaConfig and tombstones retention policy (#420)
  • Sync Action attributes with delta (#380)
  • Add LICENSE file in the Python binding and refer it in the pyproject.toml (#422)
  • Change checkpoint creation logs from info to debug (#423)
  • Add the Glue Data Catalog for reading the DeltaTable (#419)
  • Add S3StorageOptions to allow configuring S3 backend explicitly (#418)
  • BUGFIX: writes to gcs must include the content length header
  • Ensures that all table schemas are of StructType (#415)
  • Fix reading nullable action fields from parquet (#417)
  • Add filesystem argument for reading DeltaTable in Python binding (#414)
  • Add implementation for load_with_datetime in Python package. (#411)
  • Add a Makefile build task in the Python binding (#410)
  • Use update_incremental in update (#398)
  • Use tokio::fs::rename in put_obj. (#403)
  • Update python readme (#406)
  • Update pyproject definition in pyproject.toml (#405)
  • Add examples for reading delta table with Rust API. (#400)
  • Implement delete_objs in fs and s3 storage backends. (#395)
  • Remove version param from create_checkpoint_from_table (#399)
  • Google cloud storage backend (#355)
  • added initial commit info on create method for a DeltaTable (#387)
  • Upgrade to DataFusion 5.0 (#389)
  • additional error handling to atomic_rename (#386)
  • Reuse table/storage instances in checkpoints (#384)
  • Add sts assume role creds for S3 (#383)
  • Update datafusion and ballista links in README (#382)
  • Merge Cargo.toml into pyproject.toml (#381)
  • Implement consistent behavior in Windows with regard to swap parameter. (#379)
  • Refactoring of black, isort, mypy tools usages into pyproject.toml (#378)
  • Wrap DeltaTransactionError with DeltaTableError. (#374)
  • Allow filesystem backend put_obj to overwrite existing (#376)
  • Make Format.options to be required field (#370)
  • Implement atomic put_obj. (#367)
  • support partition value string deserialization for float/double/date (#363)
  • Add '.tmp' suffix to temporary file of prepared commit (#366)
  • cache cargo builds in CI (#359)