Repository navigation
Releases: jose-pr/objpath
Release list
v0.1.1
Added
rm()andcopy()takemax_workers, the keyword ofpathapi's recursive
rm()andcopy()(a class runs serially unless it declares_threadsafe,
which these classes do not). A streamed copy passes it on; the batch delete and the
store's own copy call ignore it. Nothing changes for a call that does not give it,
and it works with thepathapirelease that lacks it as long as it is not given.
Full Changelog: v0.1.0...v0.1.1
v0.1.0
The first release of objpath: the Amazon S3, Google Cloud Storage and Azure Blob
Storage support of pathlib-next 0.9.14 (pathlib_next.uri.schemes.s3, .gs and
.az) as a distribution of its own, with paths that need no URI. It needs Python 3.9 or
newer and pathapi>=0.1.0,<0.2. No store SDK is required to import it: the extras
s3 (boto3>=1.36.0,<2), gs (google-cloud-storage>=3.0,<4) and az
(azure-storage-blob>=12.4.0,<13 and azure-identity>=1.0,<2) add one, and uri
(pathapi-uri>=0.1.0,<0.2) adds the s3:, gs: and az: schemes. Under Changed, the
migration from pathlib_next.uri.schemes comes in this order: the table of names, the
table of extras, then every change of behaviour with what to do about it.
It is built on pathapi, which was published as pathlib-next up to 0.10.0 (import
name pathlib_next); the old names below are the ones before the move.
Added
S3Path,GSPathandAZPath,pathapi.Pathclasses for the objects of a
bucket or container, built from keywords:
S3Path("dir/key", bucket="bkt"),GSPath("dir/key", bucket="bkt"),
AZPath("dir/key", container="cont", account="acct"). Nothing connects until the
first request. Client options (region_name=,project=,credential=, ...) build
the path's own backend,backend=shares one between paths built separately (giving it
together with a client option is aTypeError), and a
path joined from another keeps its bucket and its backend (bucket=orcontainer=
on a joined path moves it to another bucket of the same backend). Paths built without
either use one default backend per process (AZPath: per account).S3Backend,GSBackend,AZBackendand their contractsBaseS3Backend,
BaseGSBackendandBaseAZBackend. Every backend is a context manager:close()
closes the SDK client it built, the next use builds another, and calling it twice is
harmless.objpath.exceptions:ObjPathError, apathapi.PathError, with
ObjectChangedErrorandUnreachableKeysError, both alsoOSError(see Changed).objpath.uri:S3URIPath,GSURIPathandAZURIPath, thes3:,gs:andaz:
schemes ofuripath, found byuripaththrough the entry pointss3,gsandazof
the groupuripath.schemes. Importingobjpath.uriwithouturipathraises an
ImportErrorthat namespip install "objpath[uri]";import objpathnever imports
uripathor an SDK.AZBackend.account: the account thataccount=, elseconnection_string=or
account_url=, names.objpath.__version__, read from the installed metadata.
Changed
-
The names.
pathlib_next.uri.schemes.s3,.gsand.azare gone with no alias.Was Now S3Path(aUriPath)objpath.S3Pathfor a path built from keywords;objpath.uri.S3URIPathforURIPath("s3://...")GsPathobjpath.GSPath;objpath.uri.GSURIPathAzPathobjpath.AZPath;objpath.uri.AZURIPathS3Backend,BaseS3Backendobjpath.S3Backend,objpath.BaseS3BackendGsBackend,BaseGsBackendobjpath.GSBackend,objpath.BaseGSBackendAzBackend,BaseAzBackendobjpath.AZBackend,objpath.BaseAZBackendGsPath.bucket_nameGSPath.bucket(andGSURIPath.bucket)the entry-point group pathlib_next.schemes, entriess3,gs,azthe group uripath.schemes, entriess3 = "objpath.uri:S3URIPath",gs = "objpath.uri:GSURIPath",az = "objpath.uri:AZURIPath" -
What to install where
pathlib-nexthad an extra.pathlib-next0.10 has nos3,
gsorazextra.Was Install now pathlib-next[s3]objpath[s3]pathlib-next[gs]objpath[gs]pathlib-next[az]objpath[az]any of them, for UriPath("s3://..."),gs://oraz://add objpath[uri](or installuripathbeside it) -
A path is built from keywords, not from a URL. Was:
S3Path("s3://bkt/dir/key")
parsed the URL, and the bucket came from its host. NowS3Path("dir/key", bucket="bkt"),
GSPath("dir/key", bucket="bkt")andAZPath("dir/key", container="cont", account="acct"); a URL is aURIPath(URIPath("s3://bkt/dir/key"), which needs the
uriextra). A path without a bucket is aTypeErrorwhere it is built; an
S3URIPathwith no bucket still raisesFileNotFoundErrorwhen it is used. -
str(path)of anS3Path,GSPathandAZPathis the URL (s3://bkt/dir/key), as
the URI paths' is.repr()isS3Path('dir/key', bucket='bkt'). -
Two paths are equal when their class, bucket (account and container, for
AZPath)
and key are;xand/xare one key. -
An
AZPathneeds one ofaccount=,backend=, a client option or anAZPathto join
from; without one it is aTypeErrorwhere it is built. Was: the account was the host of the
URL, and a URL without one raisedFileNotFoundErrorwhen it was used (anAZURIPathstill
does). For URI paths nothing else changes:backend=is used as it is for the path it is
given to, whatever bucket the URI names. -
The account level of Azure is a URI feature. Was:
AzPath("az://account")listed the
containers of the account. NowAZPathalways names a container;URIPath("az://account")
is still a directory whose children are the containers. -
A stream whose object was replaced while it was being read raises
ObjectChangedError,
and a copy or removal of a prefix that holds keys a walk cannot reach raises
UnreachableKeysError. Was: a plainOSError(ESTALE,EINVAL). Both are still
OSErrorwith the sameerrno, soexcept OSErroranderrnochecks keep working. -
AZBackend(connection_string=...)andAZBackend(account_url=...)carry the account they
name inaccount;accountwasNonefor them. -
The Azure tuning settings. Was: the module settings
schemes.az.READ_CHUNK(512 KiB per
ranged read request) andschemes.az.COPY_POLL_TIMEOUT(300 s to wait for a service copy).
Now: the class attributesAZBackend.read_chunkandAZBackend.copy_timeout
(BaseAZBackendholds the defaults); assign them onAZBackendor on a subclass of it. -
is_local()of a store path isFalsewithout resolving anything. Was: the URI path resolved
the bucket or account name as if it were a host name, soPathSyncer(quick_check=True)made a
DNS lookup of it and read a name that resolved to this machine as local. -
A write stream of a store that is garbage collected without being closed uploads nothing
and emits aResourceWarning;pathlib-nextclosed it, and so uploaded it. What to do:
close the file, or usewith.
Full Changelog: https://github.com/jose-pr/objpath/commits/v0.1.0