Repository navigation
v0.9.0
Added
UriPath.host_fspath(), and__fspath__()now succeeds for schemes
with_host_filesystem_path = True(currentlysftp:) instead of
unconditionally raisingNotImplementedErrorfor any non-file:
scheme.os.fspath()has two consumers -- "open this locally" (where a
remote path would silently read the wrong file) and "build a command
line for a process that runs on the path's host" (subprocess, remote
executors) -- and only the second is safe for a scheme likesftp:.
host_fspath()is the unambiguous accessor for that second case: it
never falls back to local-path semantics the way__fspath__does for
thefile:branch.progresscallback forPath.copy()/BinaryOpen.copy().BinaryOpen.copy(target, *, progress=None, chunk_size=shutil.COPY_BUFSIZE)
now streams in caller-sized chunks and, whenprogressis given, calls
progress(bytes_copied, total_size)after each chunk (total_sizeis the
source'sstat().st_sizewhen available, elseNone).Path.copy(target, ..., progress=None)wraps this with per-file identity:
progress(path, bytes_copied, total_size), so arecursive=Truecopy
reports which file is being streamed alongside its byte progress, not just
an anonymous byte stream.progress=None(the default) is behaviorally
identical to before this change -- no per-chunk overhead, same
shutil.copyfileobjbytes-on-wire.SftpPath's asyncssh concurrent
fan-out (copy(recursive=True)) does not invokeprogress-- native/batch
transfer paths are out of scope for this first cut (see
docs/divergences.md's "Deliberate extensions" section).PathSyncercan now create symlinks ontargetinstead of always
rejecting a symlink source. New constructor kwargsymlink_mode
("preserve"default,"reject"opt-out), consulted only when
follow_symlinks=Falseandsource.is_symlink()(with the default
follow_symlinks=True, symlinks are still resolved during traversal,
unchanged)."preserve"creates a matching symlink ontargetusing the
exact raw, unresolved target stringreadlink()returned -- dangling
links and relative targets included, never validated or resolved against
source's parent. Iftarget's implementation has nosymlink_to()at
all (every backend exceptLocalPathandSftpPath),"preserve"mode
raisesNotImplementedErrorthrough the existingignore_error/hook()
flow, same as every other sync branch -- not a silent skip. New
SyncEvent.Symlinkenum member.- Optional backend-native checksum protocol
(pathlib_next.protocols.checksum.NativeChecksum,
checksum(algorithm="md5") -> str). APathsubclass may implement it to
compute a file digest server-side instead of streaming the content
throughopen("rb")-- implemented onSftpPathagainst the OpenSSH
check-file@openssh.comSFTP protocol extension (paramiko backend only;
the asyncssh backend has no equivalent client-library support and
correctly falls back). Not part of the basePath/PathnameABC -- a
plainPathhas no.checksumattribute at all. Implementations MUST
raiseNotImplementedError(never return a value) when they can't
produce a genuine digest under the requested algorithm -- this is what
keeps two checksums from ever being compared under a mismatched
algorithm, or trusting something hash-shaped but not a real content hash
(e.g. S3's ETag for a multipart upload, deliberately not implemented here
for exactly that reason -- seedocs/divergences.md). Also adds a
companionsupported_checksums() -> frozenset[str]advisory query
(defaultfrozenset(), never raises);SftpPath.supported_checksums()
is a real per-connection probe against the server (paramiko exposes no
cheaper way to know), cached per connection. PathSyncer's default checksum policy now prefers native digests on
both sides when available, falling back to streaming
(utils.checksum.md5/the new genericutils.checksum.stream) when
either side can't produce one under the same algorithm -- never a
native-vs-streamed comparison under a mismatched algorithm. A
caller-suppliedchecksumcallable is unaffected (invoked exactly as
before). Newutils.checksum.native(path, algorithm) -> str | None
helper: tries the protocol, returnsNone(never raises) if unsupported.PathSyncer(..., quick_check=True)(new constructor kwarg, default
True): for a sync pair where at least one side is non-local, a
metadata-only pre-check (st_size+st_mtime, already-cached, no extra
round trip) skips the checksum call entirely when both match; a mismatch
always falls through to a real checksum rather than being treated as
"changed" on its own. Local-to-local pairs never engage this pre-check.
quick_check=Falserestores always-checksum behavior for non-local pairs.
Changed
PathSyncerdefault behavior change:PathSyncer(follow_symlinks= False).sync()on a symlink source previously always raised
NotImplementedError. It now creates a matching symlink ontargetby
default (symlink_mode="preserve"); passsymlink_mode="reject"to
restore the old unconditional-raise behavior exactly.- The
uriextra now requiresnetimps>=0.2.0(alongsideuritools).
Source.is_local()(uri.source) delegates its "is this address mine"
check tonetimps.is_local_address(), which enumerates real network
interfaces (netimps.get_interfaces()) instead of the previous
socket.getaddrinfo(socket.gethostname(), None)-based approach, which
missed addresses not tied to the resolvable hostname (VMs, containers,
VPN interfaces, additional NICs on a multi-homed host). The
hostname->address step now usesnetimps.resolve()(both"a"/"aaaa"
record types;hostis local if ANY resolved address is) instead of
socket.gethostbyname()--resolve()'s default backend chain
(dnspython, then the OS resolver viagetaddrinfo(), thennslookup)
covers the same hosts-file/NSS/DNS resolutiongethostbyname()did, and
additionally never raises for a name that simply doesn't resolve
(resolve()'s contract: always a list, empty on genuine failure) --
gethostbyname()raisedsocket.gaierrorfor that case.utils. get_machine_ips()(the old implementation's helper, unused elsewhere in
this library) is removed.SftpPath's two backends andFtpPathnow
resolve their default ports (22, 21) vianetimps.get_default_port()
instead of three separately hardcoded literals.
Fixed
Source.is_local()crashed on a bare IPv6-literalhoststring.
Source(scheme, userinfo, "::1", port)(a supported direct-construction
pattern --Source's fields are publicNamedTuplefields) raised
socket.gaierrorinstead of returning a result, because
socket.gethostbyname()(this method's original hostname-resolution
step) is IPv4-only.hostbracket-literals arriving via the normal
Source.from_str()/Uri()construction path were unaffected
(_decode_host()already parses those into a realIPv6Addressbefore
is_local()ever sees a string).is_local()now tries
netimps.try_parse()first (handles any IP literal directly, string or
not) before falling through to hostname resolution.uri.source.Sourceleaked the password instr()/repr().
Source.__str__()calleduricompose()with the rawuserinfo
(password included) -- a genuinely valid, connectable URI string, not
just a debug rendering -- andSourcehad no custom__repr__at all,
so theNamedTupledefault rendered every field verbatim too.repr()
is what a traceback frame renders, so aSourceanywhere on a failing
call stack leaked the credential into logs, even thoughUri.__str__()
already redacted. Both now redact the password fromuserinfothe same
wayUri.__str__()does; the actual data (.userinfo,
.parsed_userinfo(),["userinfo"]) is unaffected, only display is
sanitized. This is a behavior change, not purely additive:
str(source)(orf"{source}") no longer reconstructs an authenticated
URI -- verified nothing in this codebase relied on that (every real
connection site reads individualSourcefields, never whole-object
str()), but a downstream caller that did would need the new
Source.as_str(sanitize=True)method instead:sanitize=True(the
default, matching__str__) redacts the password;sanitize=Falseis
the full, credentialed round trip -- same name/kwarg as
Uri.as_uri(sanitize=), so both classes work the same way.
Full Changelog: v0.8.6...v0.9.0