Skip to content

1.2.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 01 Oct 17:28
1.2.0
0f4fe18
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Highlights

  • Workflow payloads live in their own tables (#579): 1.2 completes the move that 1.1 started. Workflow inputs, outputs and errors are now written to workflow_input and workflow_output, the same as Python and TypeScript. Migrations now go up to 114.
  • The debouncer now uses delayed workflows (#594): the debounced workflow itself now waits DELAYED on its queue, and each later call updates that one row. A 1.1 debouncer workflow still holding a key is taken over.
  • Child workflows inherit the parent's deadline, not its timeout (#587): a queued child can no longer outlive its parent. This matches Python and TypeScript.
  • System-database timestamps come from the database clock (#590), as in Python, TypeScript and Go. Executors on different hosts no longer write rows on different clocks.
  • Batched dequeue for partitioned queues (#595): when every partition runs one workflow at a time, the listener claims the head of every idle partition in one transaction.

Upgrading from 1.1

Read this section before rolling 1.2 out to a running fleet.

  • Upgrade the whole fleet to 1.1 before moving to 1.2. 1.2 finishes the two changes that 1.1 split across releases. 1.1 reads both the old and the new format, and 1.2 now writes the new one. 1.1 and 1.2 can share a fleet, and you can roll back from 1.2 to 1.1. Don't run 1.0 and 1.2 in the same fleet, and don't roll a 1.2 deployment back to 1.0. Migrations apply to the whole database as soon as the first 1.2 node starts.
    • Workflow inputs and outputs (#579). New rows store their payloads only in workflow_input and workflow_output. A 1.0 node reads only the old workflow_status columns, so it would see these payloads as missing.
    • Debouncer (#594). 1.2 writes debounced workflows as DELAYED rows. 1.1 can read and extend these rows; 1.0 cannot.
  • Migrations go up to 114. Migration 113 changes the enqueue_workflow SQL function so it writes inputs to workflow_input. The new function body is identical to Python's and TypeScript's. Migration 114 drops idx_notifications, which duplicated idx_workflow_topic, and runs online. The minimum supported schema version is still 111.
  • Leftover 1.1 debouncer workflows. A 1.1 debouncerWorkflow that still holds a key is taken over by the next debounce on that key, after about 5 seconds. If your application doesn't pin its version, cancel any leftover debouncerWorkflow workflows whose keys will never be debounced again.
  • Don't import a 1.2 export into 1.1.0 or 1.0. It is accepted without any error, but it loses its payloads. Those releases ignore the new payloads field, so the workflow's inputs, output, error and serialization import as NULL, and so do each step's output and error. Exports move safely between 1.2 and 1.1.1 in both directions. An export from 1.1.0 or earlier imports correctly into 1.2.

New features

  • Debouncer.withTimeout(Duration) (#594). Sets a timeout for every workflow the debouncer starts, counted from when the workflow is dequeued. It takes precedence over a timeout set with WorkflowOptions around the call. That timeout still applies when this option isn't set.

  • isFork workflow filter (#593). ListWorkflowsInput.withIsFork(Boolean) lists only forks (true) or only workflows that aren't forks (false). wasForkedFrom covers the other end of a fork. Conductor's list_workflows and list_queued_workflows accept is_fork.

  • Plain-value QueueOptions builders (#589). Start from new QueueOptions(), which leaves every property unset, then chain:

    • withConcurrency(Integer), withWorkerConcurrency(Integer)
    • withRateLimit(Integer, Duration), withRateLimit(int, long, TimeUnit)
    • withPartitionConcurrency(Integer), withPartitionWorkerConcurrency(Integer)
    • withPartitionRateLimit(...), in the same two forms as withRateLimit
    • withPollingInterval(Duration)
    // before
dbos.registerQueue("email", QueueOptions.setConcurrency(10).andRateLimit(100, Duration.ofSeconds(60)));

// after
dbos.registerQueue("email", new QueueOptions().withConcurrency(10).withRateLimit(100, Duration.ofSeconds(60)));

    On updateQueue, withRateLimitMax(Integer) and withRateLimitPeriod(Duration) change one half of a stored rate limit and keep the other half. Their partition versions work the same way. Setting half a limit is refused at registration, and on a queue that has no stored limit.

    Passing null clears a limit on updateQueue. While the deprecated Field overloads still exist, write it with a cast, as in withConcurrency((Integer) null).

  • WorkflowStatus.isDebounced() and debounceDeadline() (#594). Export and import carry both. An export that doesn't have them imports as not debounced.

Behavior changes

  • Debouncer (#594).

    • The debounced workflow is written directly as a DELAYED workflow on its queue. It holds workflowName-key as its deduplication ID, and its delay is capped at the debounce deadline. Without a queue, it goes on the internal queue instead of being started directly.
    • A mixed 1.1/1.2 fleet still runs every call on a key as one execution. When a 1.1 debouncer workflow holds the key, 1.2 forwards the call to it, as 1.1 did.
    • The debouncer still replays debounces that 1.1 recorded.
    • withDeduplicationId is now ignored on both debouncers, as announced when 1.1 deprecated it.
    • A negative priority, or a timeout of zero or less, is now rejected when you call the setter, not at debounce(). A priority without a queue is still rejected at debounce().

  • How a child workflow's time limit is chosen (#587). A child uses the first of these that is set:

    1. what the call itself gives (StartWorkflowOptions or EnqueueOptions): a deadline, a timeout, Timeout.none() or Timeout.inherit();
    2. what a WorkflowOptions block around the call gives;
    3. the parent's deadline.

    Before, a queued child copied its parent's timeout and started a fresh copy of it on dequeue, so it could outlive its parent. A queued child that is dequeued after its inherited deadline has passed is cancelled. Other effects:

    • A timeout, deadline, none() or inherit() given for a call replaces the whole bound set by WorkflowOptions. Before, the fields were merged one at a time, so a deadline from WorkflowOptions could override a timeout given for the call.
    • An inner WorkflowOptions block that sets a timeout, deadline or none() replaces the outer block's timeout and deadline together. The outer values return when the inner block closes.
    • A child that inherited its bound has a null workflow_timeout_ms. This shows in listWorkflows and in Conductor, and DBOSContext.getTimeout() returns null inside that child.
    • Resuming a child that inherited only a deadline leaves it with no bound, because resume clears the deadline and keeps the timeout.
    • The internal debouncer workflow always runs without a timeout. Outside a workflow, a timeout set with WorkflowOptions around debounce now limits your workflow, not the internal one.
    • Building a StartWorkflowOptions with both an explicit timeout and a deadline now throws when the options are built, as EnqueueOptions already did. Before, startWorkflow threw, and DBOSIntegration.startRegisteredWorkflow silently dropped the timeout.

  • Database clock for workflow_status and queues times (#590). These are now stamped with the database's now(), not the JVM clock:

    • created_at, updated_at and completed_at;
    • the dequeue's started_at_epoch_ms;
    • rate-limit windows and recovery.

    Before, executors sharing a system database stamped rows with their own host clocks, so FIFO order, rate limits and retention cutoffs were affected by clock skew between hosts. Delays, a queued workflow's deadline, step timings, durable sleeps and in-memory timers still use the JVM clock, as in the other SDKs.

  • Batched dequeue for partitioned queues (#595). The batched claim applies to queues that meet both conditions:

    • partition concurrency is 1, or the queue is a legacy partitionQueue queue with concurrency 1;
    • there's no queue-wide concurrency, rate limit or partition rate limit.

    For these queues, one transaction claims up to 8192 partition heads, in partition-key order, limited by the worker's budget. Other partitioned queues still use one transaction per partition. Listing a queue's partitions no longer reads every ENQUEUED row.

  • Payload writes (#579).

    • Start writes the inputs only when that call created the workflow. Completion writes the output only if the move out of PENDING succeeded, so a refused outcome leaves no orphan payload row.
    • Fork copies the inputs under the new workflow IDs.
    • recordErrorForUnstartedWorkflow is now a single transaction.

  • An unregistered workflow passed to DBOSIntegration.runWorkflow throws DBOSWorkflowFunctionNotFoundException (#581), naming the workflow. Before, it threw NoSuchElementException: No value present.

Fixes

  • A Java executor that dequeued a workflow enqueued without a class name threw NullPointerException (#581). Such a workflow targets SDKs that look workflows up by name alone, such as Python. Now it fails with DBOSWorkflowFunctionNotFoundException, and the row stays PENDING, the same as for any unregistered workflow.
  • Exporting a workflow through Conductor and importing it again lost the Java types of its payloads (#583). Inputs, output and step outputs came back as LinkedHashMap, and the error came back as a plain RuntimeException. Exports now carry the stored strings unchanged. A workflow in a format this application can't read, such as py_pickle, can now be exported too.
  • Importing a workflow could pick up rows left under the same IDs by an earlier workflow that retention had collected but not yet swept (#583). For example, a pending import showed the old output as its own, or failed on a step number already in use. Import now clears those rows first.
  • Migration 1 installed the uuid-ossp extension, which nothing uses (#580). Installing it needs CREATE on the database, so a role granted only USAGE, CREATE on a pre-created DBOS schema couldn't migrate a new system database. Databases that already ran migration 1 keep the extension.
  • On replay, a debounce that had lost the race to create its workflow ran the enqueue again for real (#594). If the key had been freed since, that started a second workflow. 1.0 and 1.1 have the same problem.
  • WorkflowStatus.equals and hashCode ignored applicationName (#594).

Deprecations

These are @Deprecated(since = "1.2", forRemoval = true). Deprecation notes across the SDK, including the 1.1 deprecations, now say "removed in a future release" instead of "removed in 2.0".

  • The old QueueOptions builders (#589): QueueOptions.empty() (use new QueueOptions()), the nine static setX factories and their andX counterparts, and every withX overload that takes a Field or Optional. Each one has a plain-value replacement.
  • Absolute workflow deadlines (#587): withDeadline and deadline() on StartWorkflowOptions, EnqueueOptions and WorkflowOptions. No other SDK lets a caller set a deadline. Inherited deadlines and WorkflowStatus.deadline() are not deprecated.

Build and tests

  • No dependency changes.
  • Test stability fixes (#510, #566, #567, #568, #570, #571). These include a five-minute timeout for the from-empty CockroachDB migration tests, and waiting for a pooled test container's previous user to disconnect before reusing it.
  • QueueService.pause() now waits up to 30 s for the poll pass in flight before returning (#569). Before, a pass that had already started could claim rows enqueued after pause() returned. Only tests call pause().

Full Changelog: 1.1.0...1.2.0

Assets 17
Loading