Skip to content

v0.3.1

Choose a tag to compare

@vertix vertix released this 30 May 21:56
· 2 commits to main since this release
Immutable release. Only release title and notes can be modified.
fcdae81

Added

  • pos3 console-script entry point with ls, download, upload subcommands
    (#10).
    • pos3 ls <prefix> [-r] [--profile NAME] lists objects, one full s3:// URL
      per line on stdout.
    • pos3 download <url> [--local PATH] [--delete] [--exclude PATTERN]... [--profile NAME]
      prints only the resulting local path to stdout; progress and logs go to
      stderr, so data_dir=$(pos3 download s3://bucket/dataset/) is safe.
    • pos3 upload <url> [--local PATH] [--delete] [--exclude PATTERN]... [--profile NAME]
      is one-shot (no background loop or interval). Source defaults to the cache
      path pos3 download would have produced; errors if the source doesn't
      exist.
    • --delete defaults OFF for both download and upload (the Python API
      defaults to True; the CLI is more conservative for interactive use).
    • --profile is supported alongside the URL form s3://<profile>@bucket/...;
      the URL form wins on conflict, matching the Python precedence.
    • -n / --dry-run on download and upload prints the planned per-file
      actions to stdout (in aws s3 sync --dryrun style) and performs no
      transfers, no deletes, and no directory creation.
    • download and upload require an s3:// URL. The Python API's
      local-path passthrough still works in code; the CLI rejects non-S3
      inputs with a clear error so a typo can't silently succeed. ls is
      unchanged and still accepts both forms.
  • pos3.TransferPlan dataclass plus module-level pos3.plan_download(remote, ...)
    and pos3.plan_upload(remote, ...) wrappers: compute the set of
    (source, destination) copies and target deletes a real call would
    perform, without performing any of them. Same calling pattern as
    pos3.download / pos3.upload — use them inside a with pos3.mirror():
    block. The CLI's -n / --dry-run is implemented on top of these.
  • TransferError and TransferPlan are now in pos3.__all__.

Changed

  • Per-object transfer failures now raise pos3.TransferError instead of
    being logged and swallowed. Previously, if any worker in a download or
    upload batch failed, the error was sent to the logger and the call
    returned normally — data_dir = pos3.download(...) could return a path
    to a partial cache, and pos3 download exited 0 after a failed S3 GET.
    Both now propagate. The new exception exposes .operation and
    .failures (list of the underlying per-worker exceptions). The CLI
    catches it and exits 1 with the failure on stderr. Background
    interval syncs
    (upload(..., interval=N)) are best-effort: a
    TransferError from one tick is logged and the daemon continues so
    the next interval can retry. Only the final sync on context exit and
    any one-shot call propagate. Cleanup on error — when the
    mirror() body is unwinding with an exception, a TransferError from
    the sync_on_error=True cleanup sync is logged but swallowed, so the
    original application exception remains the visible cause.
  • pos3.mirror() no longer creates the cache root directory eagerly on
    context entry. The leaf directory is still created on demand when a
    file is actually downloaded, so the visible behavior of download() is
    unchanged. Dry-run and plan_* paths are now genuinely side-effect
    free on the local filesystem.