1.2.0

Multi-Storage Client (MSC)

This release adds read-only replicas and bundled typing support, with fixes for transfers, synchronization, caching, and storage providers. The AIStore extra now requires SDK 2.x.

Breaking Changes

• Dependency compatibility: the aistore extra now requires AIStore Python SDK 2.x (aistore>=2.0,<3) and no longer supports SDK 1.x. Applications that pin aistore<2 must update their dependency constraints; applications that also call the SDK directly may need to migrate those calls to SDK 2.x. MSC's AIStore provider APIs and configuration remain unchanged.

New Features

• Added read-only replicas with read_only: true. MSC reads from these replicas in priority order but skips them for upload-on-miss, source deletions, and default replica synchronization.
• Added a PEP 561 py.typed marker so type checkers can use MSC's bundled type annotations.

Bug Fixes

• Presigned URLs now resolve logical paths through the configured metadata provider before signing the physical storage path.
• Corrected public type hints for file-like upload/download arguments and sequence-based batch operations and replica indices.
• S3-over-RDMA transfers no longer use the HTTP 100-continue handshake.
• Rust-backed S3 and GCS transfers now apply read_timeout to each read, resetting after successful reads, with a 60-second default matching boto3. The separate total HTTP request timeout is 600 seconds (per part for multipart downloads), avoiding premature timeouts during slow but active transfers.
• Rust-backed batch downloads and uploads now correctly classify transient Rust errors as retryable, allowing MSC to retry failed files.
• sync_from(..., source_files=[...]) now honors symlink_handling for explicitly selected POSIX file symlinks instead of always preserving them.
• Rust-backed S3 and GCS uploads now include user-defined object attributes as metadata. Invalid HTTP metadata headers produce a descriptive error instead of panicking.
• Path.is_relative_to() returns False for paths in different storage profiles, even when their internal paths overlap, consistent with Path.relative_to().
• GCS uploads and downloads now always remove their staging temp file when the transfer fails. Previously every failed attempt leaked the file, which for downloads to a path is created in the destination directory: a partial file for interrupted transfers, or at least multipart_threshold bytes for a failed multipart upload.
• Compare normalized paths in the sync_from overlap check, so /data vs data/sub on the same profile is rejected.
• S3 and OCI listings with include_directories=True and end_at no longer drop in-range objects when the same page also contains a directory prefix that sorts after end_at.
• Parallel recursive listing no longer issues a duplicate LIST request for a prefix that was fetched directly while still queued for prefetching.
• download_file now hands the resolved symlink target's metadata to the provider (instead of the zero-length symlink marker's), so large targets use multipart downloads and READ metrics report the real size; it also no longer issues two HEAD requests when no metadata is supplied.
• sync / sync_from with delete_unmatched_files=True now reports the correct target key for pruned objects in subdirectories. When the target has a metadata provider (e.g. manifest), the sync previously failed with SyncError: Object ... does not exist and, in multi-process mode, committed a manifest that still listed the deleted objects.
• HuggingFace info() and list() report the last commit date as last_modified again instead of always returning the epoch sentinel.
• A multi-process sync that stops early on an error (fast-fail) no longer hangs the Python process at exit while flushing unconsumed work batches.
• Report delete_many failures from sync DELETE batches on the error queue (previously the exception killed the worker thread and the sync reported success).
• Full-file read(path, byte_range=Range(0, size)) calls with caching enabled now serve repeat reads from the cache instead of re-downloading and re-caching the whole object every time.
• open(path, "r", encoding=...) with caching enabled now honors the requested mode and encoding when the object is served from the cache; the cache-first fast path previously opened the cached file in binary mode (so read() returned bytes), and cached text opens ignored a non-default encoding.
• Stop counting pattern-excluded files that exist on both sides with differing metadata as sync work units; they inflated the progress-bar total and SyncResult.total_work_units.
• Cache access-time updates no longer raise when the cached file was concurrently evicted or is owned by another user, and they operate on an open file descriptor so a concurrently recreated cache file is never chmod'ed by mistake; the permission restore in the finally block previously re-raised the error the method was meant to swallow.
• Return copies from list(..., include_url_prefix=True) and the single-file path of list() instead of rewriting the key of ObjectMetadata objects owned by the metadata provider; a single msc.list() previously corrupted the in-memory manifest keys.
• HuggingFace listings now raise FileNotFoundError for a missing repository or revision instead of silently returning no objects; only a missing directory yields an empty listing.
• S3 uploads, append-on-close, and copies denied with HTTP 403 now consistently raise PermissionError, including multipart uploads.
• s3_cuobject (S3-over-RDMA) now sends the cuObject descriptor as x-amz-rdma-token unchanged and mints a new token for every retry. It validates x-amz-rdma-reply strictly against the HTTP status and honors x-amz-rdma-bytes-transferred. Transfers larger than rdma.multipart_chunksize (at most 4 GiB) are split into multipart uploads or ranged GETs, where earlier they failed or sent too much in one transfer.