Skip to content

0.47.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Apr 21:15
· 47 commits to main since this release
0.47.0
46382a1
This commit was signed with the committer’s verified signature.
shunjiad Shunjia Ding
GPG key ID: 1D3C72C8943CDD1E
Verified
Learn about vigilant mode.

Multi-Storage Client (MSC)

Breaking Changes

  • s8k provider no longer pins request_checksum_calculation/response_checksum_validation to when_required; botocore defaults now apply, and botocore >= 1.36.0 adds a CRC32 checksum to every upload.

New Features

  • Added checksum_algorithm option to s3 and s8k providers for AWS S3 additional upload checksums (CRC32, CRC32C, SHA1, SHA256, CRC64NVME). When rust_client is enabled, only SHA256 is supported.
  • End-to-end symlink preservation for uploads and downloads. MSC now treats symbolic links as a first-class concept across all storage providers, so symlinks in a POSIX source tree can be round-tripped through object storage without losing their link semantics. This adds:
    • A new make_symlink API on every StorageProvider (S3, GCS, Azure, OCI, AIS, POSIX), which writes an empty object whose user metadata carries the link target on cloud backends, and a native symlink on POSIX.
    • A symlink_handling option on list / list_files with three modes: FOLLOW (default, backward-compatible), SKIP (exclude symlinks), and PRESERVE (yield symlinks as leaf entries with ObjectMetadata.symlink_target populated).
    • sync_from / msc sync honor symlink_handling=PRESERVE to recreate symlinks at the destination instead of dereferencing them, so uploading a directory to object storage and downloading it back reconstructs the original link structure.
  • Add opt-in client-side MD5 verification to the Azure storage provider.
  • Support multipart upload and download in the Azure storage provider.

Bug Fixes

  • Sort directory entries in the POSIX storage provider to match S3 lexicographical order.
  • Improve msc sync performance by skipping per-object metadata lookups for files up to 16MB, avoiding redundant HEAD requests during synchronization.
  • Upgrade fastmcp, cryptography, pygments, and axios to address security advisories.

Multi-Storage File System (MSFS)

New Features

  • Add fio benchmark script for MSFS performance testing.

Bug Fixes

  • Add globalsStruct lock tracking.
  • Additional fixes for the fio benchmark.
Assets 10
Loading