Skip to content

v0.28.0

Latest

Choose a tag to compare

@sprocketbio sprocketbio released this 15 Jul 19:31
5b79a1a

This release lands the foundational analysis support for Sprocket modules that we've been building toward, brings a full management surface to sprocket dev server, adds a new call-caching mode, and ships a batch of performance, tooling, and error-message improvements. Note that it includes a few breaking changes to the sprocket dev server and sprocket dev test command surfaces—see the highlights and changelog below.

  • Manage runs from the command line. sprocket dev server now has status, inspect, cancel, and retry subcommands, backed by new HTTP API endpoints for run and task introspection (#915). Heads up: the server commands were regrouped under sprocket dev server—the old server subcommand is now start, and submit moved under it as well.
  • Resolve module imports during analysis. Analysis can now resolve symbolic module imports (import owner/module/path, including the import * from … and import { a, b } from … forms) through a wdl-modules resolver, materializing them to concrete files—the groundwork for full module support (#872).
  • A new call-caching strategy. The strongish content digest mode hashes file size, last modified time, and the first 10 MiB of contents—a middle ground between weak and strong, similar to Cromwell's fingerprint strategy (#978).
  • Faster and lighter. The global allocator moved from peak_alloc to mimalloc. peak_alloc's per-allocation atomic bookkeeping made the analysis benchmark about 66% slower than the system allocator; with that overhead gone and mimalloc in its place, the parse-and-analyze path measured about 17% faster than the system allocator. Peak memory is now queried once on exit instead (#990). This should result in faster workflow execution for most users.
  • Clearer diagnostics. Better hints when a scalar input receives multiple values, version-aware runtime key type checking (WDL 1.0 vs 1.1), and a corrected "missing version statement" error that no longer misinforms draft-2 users. New lint/analysis rules land too: BashSetSyntax, MeaninglessLintDirective, and KnownRules.

What's Changed

sprocket

Added

  • Added server-management commands under sprocket dev server (#915).
    • status shows a compact one-line status for a specific run, or lists all runs with pagination and status filtering; supports --json.
    • inspect shows detailed information about a run, including per-status task counts, the server's output directory, and (with --detailed) a per-task breakdown.
    • cancel cancels a queued or running run.
    • retry resubmits a previous run with optional input overrides using the same syntax as submit (key=value, @file, repeated keys append to arrays).
  • New API endpoints on the dev server HTTP API: GET /api/v1/info (server metadata), GET /api/v1/runs/{id}/tasks (list a run's tasks, paginated and filterable by status), GET /api/v1/runs/{id}/tasks/counts (per-status task counts), GET /api/v1/tasks (list all tasks), GET /api/v1/tasks/{name} (single task), and GET /api/v1/tasks/{name}/logs (task logs, filterable by stdout/stderr) (#915).
  • Added a strongish content digest mode (run.task.digests = "strongish") that hashes file size, last modified time, and the first 10 MiB of a file's contents; this is an intermediate strategy between weak and strong, similar to Cromwell's fingerprint call caching strategy (#978).
  • Added -t (--target), -f (--filter), and --exact options to sprocket dev test (#952).
  • Added sprocket dev test schema subcommand to generate a JSON schema for Sprocket test definitions (#953).
  • Added sprocket config schema subcommand to generate a JSON schema for sprocket.toml files (#958).

Changed

  • Grouped the server commands under sprocket dev server (previously flat under sprocket dev): server and submit are now sprocket dev server <subcommand>. The server subcommand was renamed start. (#915).
  • sprocket analyzer now honors [format] configuration (#986).
  • Replaced the peak_alloc global allocator with mimalloc and now query peak memory usage from the operating system on exit, removing per-allocation tracking overhead (#990).
  • Errors reported for a scalar input (e.g., File) that received multiple values now include a hint pointing to the likely cause—a repeated key=value on the command line or an unquoted shell glob (e.g., key=*.txt) that expanded to more than one value (#998).
  • The -t (--include-tag) and -f (--filter-tag) options for sprocket dev test have been renamed to -i and -e (--exclude-tag), respectively (#952).

Fixed

  • dev server HTTP endpoints and CLI commands now reject non-positive --limit values and negative or unparsable next_token values (previously limit=0 could loop pagination and limit=-1 triggered SQLite's unbounded-fetch behavior) (#915).
  • The "missing version statement" parse error no longer claims all WDL documents require a version statement (untrue for draft-2); it now scopes the claim to WDL v1.0+ and adds migration guidance for draft-2 documents (#993).

wdl-grammar

Added

  • Added the unstable-python feature flag, which enables APIs related to Sprocket's Python bindings (#941).

Fixed

  • Diagnostic now considers its help field in its Ord implementation (#941).
  • The preamble parser's "missing version statement" diagnostic no longer overclaims that all WDL documents require a version statement; it is now scoped to WDL v1.0+ and includes migration guidance for draft-2 documents (#993).

wdl-ast

Added

  • Added the unstable-python feature flag, which enables APIs related to Sprocket's Python bindings (#941).

wdl-analysis

Added

  • Analysis resolves symbolic module imports (import owner/module/path, including the wildcard import * from owner/module and selected-member import { a, b } from owner/module forms) through a wdl-modules Resolver, materializing them to concrete files during analysis (#872).
  • MeaninglessLintDirective rule, which flags #@ except comments that don't suppress anything (#858).
  • KnownRules rule, which ensures only known rules are used in except directives (#858).

Changed

  • Analyzer::new and Analyzer::new_with_validator now take a ResolutionContext (a resolver plus an optional manifest path) in place of separate arguments; pass ResolutionContext::default() to preserve the previous non-resolving behavior (#872).
  • The analyzer's formatter now honors [format] configuration (#986).

Fixed

  • runtime section key type checking (e.g. for cpu, gpu, disks, maxRetries, and returnCodes) is now version-aware: these keys are no longer type checked in WDL 1.0 documents, since they were not formally typed until WDL 1.1 (#811).

wdl-lint

Added

  • Added BashSetSyntax lint rule to enforce consistent bash set commands in every command section (#843).

Removed

  • Removed the KnownRules rule, and promoted it to wdl-analysis (#858).

wdl-engine

Added

  • Added a strongish content digest mode (run.task.digests = "strongish") that hashes file size, last modified time, and the first 10 MiB of a file's contents; this is an intermediate strategy between weak and strong, similar to Cromwell's fingerprint call caching strategy (#978).

Changed

  • Input type-mismatch errors now include a hint when a scalar input (e.g., File) was given an array value, explaining that this commonly happens when a key is repeated on the command line or when an unquoted shell glob expands to more than one value (#998).

wdl-diagnostics

Added

  • Added the unstable-python feature flag, which enables APIs related to Sprocket's Python bindings (#941).

wdl-lsp

Added

  • Added support for the textDocument/codeLens request on tasks/workflows with no required inputs (#981).

Changed

  • [format] configuration now honored when formatting documents, matching sprocket format output (#986).