Skip to content

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 29 Jul 05:38
· 432 commits to main since this release

Added

  • UriPath.host_fspath(), and __fspath__() now succeeds for schemes
    with _host_filesystem_path = True (currently sftp:) instead of
    unconditionally raising NotImplementedError for any non-file:
    scheme. os.fspath() has two consumers -- "open this locally" (where a
    remote path would silently read the wrong file) and "build a command
    line for a process that runs on the path's host" (subprocess, remote
    executors) -- and only the second is safe for a scheme like sftp:.
    host_fspath() is the unambiguous accessor for that second case: it
    never falls back to local-path semantics the way __fspath__ does for
    the file: branch.
  • progress callback for Path.copy()/BinaryOpen.copy(). BinaryOpen.copy(target, *, progress=None, chunk_size=shutil.COPY_BUFSIZE)
    now streams in caller-sized chunks and, when progress is given, calls
    progress(bytes_copied, total_size) after each chunk (total_size is the
    source's stat().st_size when available, else None). Path.copy(target, ..., progress=None) wraps this with per-file identity:
    progress(path, bytes_copied, total_size), so a recursive=True copy
    reports which file is being streamed alongside its byte progress, not just
    an anonymous byte stream. progress=None (the default) is behaviorally
    identical to before this change -- no per-chunk overhead, same
    shutil.copyfileobj bytes-on-wire. SftpPath's asyncssh concurrent
    fan-out (copy(recursive=True)) does not invoke progress -- native/batch
    transfer paths are out of scope for this first cut (see
    docs/divergences.md's "Deliberate extensions" section).
  • PathSyncer can now create symlinks on target instead of always
    rejecting a symlink source.
    New constructor kwarg symlink_mode
    ("preserve" default, "reject" opt-out), consulted only when
    follow_symlinks=False and source.is_symlink() (with the default
    follow_symlinks=True, symlinks are still resolved during traversal,
    unchanged). "preserve" creates a matching symlink on target using the
    exact raw, unresolved target string readlink() returned -- dangling
    links and relative targets included, never validated or resolved against
    source's parent. If target's implementation has no symlink_to() at
    all (every backend except LocalPath and SftpPath), "preserve" mode
    raises NotImplementedError through the existing ignore_error/hook()
    flow, same as every other sync branch -- not a silent skip. New
    SyncEvent.Symlink enum member.
  • Optional backend-native checksum protocol
    (pathlib_next.protocols.checksum.NativeChecksum,
    checksum(algorithm="md5") -> str). A Path subclass may implement it to
    compute a file digest server-side instead of streaming the content
    through open("rb") -- implemented on SftpPath against the OpenSSH
    check-file@openssh.com SFTP protocol extension (paramiko backend only;
    the asyncssh backend has no equivalent client-library support and
    correctly falls back). Not part of the base Path/Pathname ABC -- a
    plain Path has no .checksum attribute at all. Implementations MUST
    raise NotImplementedError (never return a value) when they can't
    produce a genuine digest under the requested algorithm -- this is what
    keeps two checksums from ever being compared under a mismatched
    algorithm, or trusting something hash-shaped but not a real content hash
    (e.g. S3's ETag for a multipart upload, deliberately not implemented here
    for exactly that reason -- see docs/divergences.md). Also adds a
    companion supported_checksums() -> frozenset[str] advisory query
    (default frozenset(), never raises); SftpPath.supported_checksums()
    is a real per-connection probe against the server (paramiko exposes no
    cheaper way to know), cached per connection.
  • PathSyncer's default checksum policy now prefers native digests on
    both sides when available
    , falling back to streaming
    (utils.checksum.md5/the new generic utils.checksum.stream) when
    either side can't produce one under the same algorithm -- never a
    native-vs-streamed comparison under a mismatched algorithm. A
    caller-supplied checksum callable is unaffected (invoked exactly as
    before). New utils.checksum.native(path, algorithm) -> str | None
    helper: tries the protocol, returns None (never raises) if unsupported.
  • PathSyncer(..., quick_check=True) (new constructor kwarg, default
    True): for a sync pair where at least one side is non-local, a
    metadata-only pre-check (st_size + st_mtime, already-cached, no extra
    round trip) skips the checksum call entirely when both match; a mismatch
    always falls through to a real checksum rather than being treated as
    "changed" on its own. Local-to-local pairs never engage this pre-check.
    quick_check=False restores always-checksum behavior for non-local pairs.

Changed

  • PathSyncer default behavior change: PathSyncer(follow_symlinks= False).sync() on a symlink source previously always raised
    NotImplementedError. It now creates a matching symlink on target by
    default (symlink_mode="preserve"); pass symlink_mode="reject" to
    restore the old unconditional-raise behavior exactly.
  • The uri extra now requires netimps>=0.2.0 (alongside uritools).
    Source.is_local() (uri.source) delegates its "is this address mine"
    check to netimps.is_local_address(), which enumerates real network
    interfaces (netimps.get_interfaces()) instead of the previous
    socket.getaddrinfo(socket.gethostname(), None)-based approach, which
    missed addresses not tied to the resolvable hostname (VMs, containers,
    VPN interfaces, additional NICs on a multi-homed host). The
    hostname->address step now uses netimps.resolve() (both "a"/"aaaa"
    record types; host is local if ANY resolved address is) instead of
    socket.gethostbyname() -- resolve()'s default backend chain
    (dnspython, then the OS resolver via getaddrinfo(), then nslookup)
    covers the same hosts-file/NSS/DNS resolution gethostbyname() did, and
    additionally never raises for a name that simply doesn't resolve
    (resolve()'s contract: always a list, empty on genuine failure) --
    gethostbyname() raised socket.gaierror for that case. utils. get_machine_ips() (the old implementation's helper, unused elsewhere in
    this library) is removed. SftpPath's two backends and FtpPath now
    resolve their default ports (22, 21) via netimps.get_default_port()
    instead of three separately hardcoded literals.

Fixed

  • Source.is_local() crashed on a bare IPv6-literal host string.
    Source(scheme, userinfo, "::1", port) (a supported direct-construction
    pattern -- Source's fields are public NamedTuple fields) raised
    socket.gaierror instead of returning a result, because
    socket.gethostbyname() (this method's original hostname-resolution
    step) is IPv4-only. host bracket-literals arriving via the normal
    Source.from_str()/Uri() construction path were unaffected
    (_decode_host() already parses those into a real IPv6Address before
    is_local() ever sees a string). is_local() now tries
    netimps.try_parse() first (handles any IP literal directly, string or
    not) before falling through to hostname resolution.
  • uri.source.Source leaked the password in str()/repr().
    Source.__str__() called uricompose() with the raw userinfo
    (password included) -- a genuinely valid, connectable URI string, not
    just a debug rendering -- and Source had no custom __repr__ at all,
    so the NamedTuple default rendered every field verbatim too. repr()
    is what a traceback frame renders, so a Source anywhere on a failing
    call stack leaked the credential into logs, even though Uri.__str__()
    already redacted. Both now redact the password from userinfo the same
    way Uri.__str__() does; the actual data (.userinfo,
    .parsed_userinfo(), ["userinfo"]) is unaffected, only display is
    sanitized. This is a behavior change, not purely additive:
    str(source) (or f"{source}") no longer reconstructs an authenticated
    URI -- verified nothing in this codebase relied on that (every real
    connection site reads individual Source fields, never whole-object
    str()), but a downstream caller that did would need the new
    Source.as_str(sanitize=True) method instead: sanitize=True (the
    default, matching __str__) redacts the password; sanitize=False is
    the full, credentialed round trip -- same name/kwarg as
    Uri.as_uri(sanitize=), so both classes work the same way.

Full Changelog: v0.8.6...v0.9.0