v0.48.0

@pipelex/sdk on npm

Added

• directory, a method's tree listed from disk: createMethod and writeDraft, like the runs, validate and the crate routes, take the method's tree as directory, a directory on disk listed in Node by the mthds package's lister under the standard's membership rule and the deployment's limits, and send it as they send a tree (docs/method-trees.md).
• CrateRouteOptions, a signal for the crate routes: resolve, codegen and pipeIo take a second argument whose signal is read before each step a directory, tree or contents costs — the version handshake, the directory's listing, the blob route, each read and upload of a missing blob — and before the request, and handed to each of those steps, so an abort stops the one under way and starts nothing further, while the route's own request, once sent, runs to its end; prepareInputs passes its own signal there.
• The files each validated step reads, typed: PipelexValidationReport declares tree_files_read, mapping each validated step's pipe_ref to the files of the method's tree it reads, each a TreeFileRead with its tree_path and its use, a TreeFileUse (template, template_asset or python_module) read open so a use a later runner adds still type-checks, and tree_file_readers, mapping each of those files to the steps that read it. Both are optional, absent on a runner predating them, and TreeFileRead and TreeFileUse are exported (docs/method-trees.md).

Changed

• Runs and validate take the method's tree (Breaking): execute, start and startAndWaitForResult, its blocking fallback included, take the method as its tree in one of four forms, directory, contents, tree or files, sent as a save sends it, and lose mthds_contents, bundle_b64 and resources, which a runner still reading them gets only through extra; a file a step reads, such as a PipeDocGen step's template, rides the tree at the path the step names. validate(source, allowSignatures?, options?) takes a PipelexValidateSource, exactly one of the tree, a method_ref or a method_id, with render, views, timeoutMs and signal in its ValidateOptions, so a call that passed inline texts, mthds_sources or positional render and views passes { files } or { contents } and the options object. Two forms of the tree, an empty files, contents or tree, a method_ref beside the tree and a suffixed method_id beside it are refused with RequestArgumentError before anything is listed or sent, and so are a directory that is not a string and a tree that is not a listed tree. A file of the tree that could not be uploaded throws MethodFileTransferError, and the run or the validation is then not sent. startAndWaitForResult prepares the tree once, under its signal, which stops the handshake, the listing, the blob route and the uploads under way so no run starts, and its fallback to the blocking execute sends the same files. A start whose body can be large, a tree, a method_ref or a bundle riding extra as mthds_contents or bundle_b64, gets the client's request ceiling rather than the 30-second start budget.
• A method's tree travels by the mthds package's rule: a directory, tree or contents, on a save as on every other call, travels by the rule the mthds package's MthdsApiClient follows: after the version handshake, inline when its map weighs no more than the lower of INLINE_TREE_MAX_BYTES and the announced max_inline_bytes, and otherwise by hash; to an API keeping no blob store it travels inline whatever its weight, and one past its max_inline_bytes is refused with RequestArgumentError before anything is sent, a tree whose files alone pass it before any file is read (docs/method-trees.md). The handshake is kept five minutes, and as mthds does, a 404 from the blob route or a 422 refusing the prepared tree's size drops it and sends once more on a fresh one, and a 422 request.blobs_missing naming blobs of the tree uploads them and sends once more; a start under an idempotency key is not sent again after a size refusal, the platform binding the key to it.
• methodTreeLimits reads a handshake at most five minutes old: the client keeps the handshake's answer five minutes for methodTreeLimits and every tree it sends, where it kept it for its lifetime, and asks again once it is older or a refusal shows it stale.
• The .mthds check hook validates the method's tree: the hook built by npm run build:hook sends the tree a run would send, the edited file's directory listed by the mthds package's lister and rooted at the nearest METHODS.toml, where it gathered the directory's .mthds files and the template files their steps name, so every file of the method reaches the dry run. It passes silently, as before when it cannot check, for a tree holding a symbolic link or a directory it cannot read, for an edited file the standard's rule keeps out of its tree, and past its caps of 100 files, 5 MiB for one file or 12 MiB in all, and the whole validation, the handshake, the blob route and the uploads included, runs under one ten-second deadline, past which it passes silently (docs/hook-bundle.md).
• The mthds floor moves from ^0.32.1 to ^0.33.0 (Breaking): what this package re-exports from mthds/protocol follows it: RunRequest carries the method's tree as a files map and loses mthds_contents and bundle_b64, assertOneMethodSource and hasMethodSource replace assertExclusiveRunSources and hasBundlePayload, and MethodTreeSource and the protocol's file-entry types are exported.
• The crate routes take the method's tree (Breaking): resolve, codegen, pipeIo and prepareInputs take the method as files, the MTHDS protocol's map from tree path to file entry, or as a directory, tree or contents sent inline or by hash as a save sends them, with dependencies, the method packages it calls by address, and read_scope beside it, which the hosted platform stamps itself and drops when a caller sends it (docs/crate-routes.md). An empty files, contents or dependencies map, and files: null, is absent and not sent, and a request carrying a read_scope asks POST /v1/blobs/missing under it, so its blobs land in the store its hash entries are read from. A file of the tree that could not be uploaded throws MethodFileTransferError, and the request is then not sent. resources is gone, a file a step reads riding the tree, and MthdsFileItem and the list of { content, source } items are gone, so a call that sent files: [{ content, source }] sends files: { [source]: content }, a prepareInputs given the old list, a contents or dependencies that is not a map of what it holds, or a tree that is not a listed tree or holds no file throws InputPreparationError before any request, and PipeIOValidReport.files echoes a map from tree path to bundle text. A request whose files carry more than INLINE_TREE_MAX_BYTES inline gets the client's request ceiling rather than the 30-second static-route budget.

Removed

• validateFiles, MthdsFile, ValidateFilesOptions, BundleResources and BundleResourceBytes (Breaking): a method's files are validated by passing its tree to validate, and the files a step reads ride that tree, so the inline-file validator and the resources types are gone, with no alias.

pipelex-sdk on PyPI

Added

• The method's tree on the run routes and validate: execute, start, start_and_wait and validate take the method as a save does, as files, the standard's map of file entries sent as given, or by name as tree or as contents; a tree or contents travels inline when small or when the API keeps no blob store, and otherwise by hash, after POST /v1/blobs/missing names the blobs the API lacks and the client uploads those, all before the request, so an upload that fails raises MethodFileTransferError, and then no run starts and no validation is sent. start_and_wait prepares the tree once, and its blocking fallback sends the same map. A tree given in two forms, beside a method_ref, or in a form of the wrong type, a files or contents that is not a map or a tree that is neither a MethodTree nor a path, is refused with RequestArgumentError before anything is listed, uploaded or sent (docs/method-trees.md).
• A directory as the method's tree: wherever a tree is taken, the runs, validate, create_method, write_draft, the crate requests and prepare_inputs, tree also takes the path of a directory or of a .mthds file, as a Path or a str, which the client lists with mthds's lister under the limits the deployment announces, so the standard's membership rule decides what is sent and a run's inputs.json or inputs.toml never is. prepare_inputs raises a refusal of that listing as an InputPreparationError refusing tree, after the version handshake that announces the limits and before the pipe I/O request.
• The inline limit the API announces: a tree is sent inline only when its inline map weighs no more than 256 KiB and no more than the API announces it takes inline (limits.method_tree.max_inline_bytes in its version handshake), and otherwise by hash, at the cost of one handshake per request, which also serves the listing of a path and a tree past the threshold. contents keeps the 256 KiB threshold alone, so a small one costs no round trip.
• The files each validated step reads, typed: PipelexValidationReport parses tree_files_read, mapping each validated step's pipe_ref to the files of the method's tree it reads, each a TreeFileRead with its tree_path and its use, a TreeFileUse member (template, template_asset or python_module) or the raw string of a use a later runner adds, and tree_file_readers, mapping each of those files to the steps that read it. Both default empty, so a body from a runner predating them still parses (docs/architecture.md).

Changed

• The run routes no longer take mthds_contents (Breaking): execute, start and start_and_wait take the method as its tree, so a call that passed mthds_contents=[bundle] passes contents={"main.mthds": bundle}, or tree=Path(directory). While the Pipelex API still reads mthds_contents, a caller that must send it passes it through extra={"mthds_contents": [...]}, which the client sends as given, with none of the checks it makes on the tree; extra no longer refuses it.
• validate takes the method's tree (Breaking): its signature is validate(files=None, allow_signatures=False, *, tree=None, contents=None, method_ref=None, method_id=None, render=None, views=None), so the method arrives as its tree in place of mthds_contents, each diagnostic naming its file by tree path, mthds_sources is gone, and render and views are taken by name. The refusal of zero or several selectors names files as the argument at fault, and an empty files is refused before sending. The hosted platform validates a tree only for a caller acting in an organization.
• The crate routes take the method's tree (Breaking): ResolveRequest, CodegenRequest and PipeIORequest, and prepare_inputs, take the method as files, the MTHDS protocol's map from tree path to FileEntry, or as a tree or contents that resolve, codegen and pipe_io send inline or by hash as a save sends them, with dependencies, the method packages it calls by address, and read_scope beside it, which the hosted platform stamps itself and drops when a caller sends it (docs/architecture.md). A request carrying a read_scope asks POST /v1/blobs/missing under it, so its blobs land in the store its hash entries are read from; a tree holding no file raises RequestArgumentError before any request; and the three request models refuse a member they do not declare, resources among them, rather than dropping it from the wire, a step's files riding in the tree itself. A file of the tree that could not be uploaded raises MethodFileTransferError, and the request is then not sent. MthdsFileItem and the list of content and source items are gone, so a call that sent files=[MthdsFileItem(content=…, source=…)] sends files={source: content}, prepare_inputs given the old list, a contents or dependencies that is not a map of what it holds, a tree that is neither a MethodTree nor a path, or holds no file, or any other selector the request refuses raises InputPreparationError before any request, and PipeIOValidReport.files echoes a map from tree path to bundle text.

Removed

• validate_files and MthdsFile (Breaking): files validated under their names are sent as the method's tree, validate({name: content, …}) or validate(tree=Path(directory)), whose diagnostics name each file by its tree path.