Skip to content

Concurrency semantics: a live job vs. the connection's declared actions #497

Description

@JArmandoAnaya

Today connection_actions is a function of setup state and connection type only, and a live job
moves neither. So a connection with an integrity check in flight still declares both
check_integrity and download_weights, and the routes accept a second request for either.
Concurrent checks are therefore possible, and so is a download overlapping a check over a single
cache. The behaviour is pinned by
test_a_live_check_does_not_change_what_the_connection_declares, which arrived with the change
that put the integrity check on the wire (#496).

It is defensible as it stands: the jobs are idempotent, and require_checkable passing at
ready is what lets an orphaned retry and a person asking twice take one path. It is also
wasteful. The download-during-check interleavings are answered only by the purge-first
crash-window ordering that came with the integrity check re-reading every byte (#475), whose
worst case lands on ready with a missing blob — a state download_weights repairs, since the
download reports its bytes and the connection carries them (#493).

Options (deferred-needs-input)

(a) Keep it as pinned. Safe by idempotency and by the ordering above; pays duplicate I/O
whenever the overlap happens.

(b) Coalesce same-type requests. A second request for a job type already in flight joins the
live job instead of forking, extending the download's existing no-fork invariant to checks.
Cross-type requests stay legal. This adds no new dimension to connection_actions, and is the
cheapest reduction of the waste.

(c) Refuse cross-type work while a job is live. connection_actions gains job-awareness. The
hidden cost is that a refusal based on job liveness has to survive a worker dying mid-job, or
else a phantom "live" job strands the connection behind actions it refuses — which is the exact
stranding the no-third-state argument set out to avoid.

Metadata

Metadata

Assignees

No one assigned

    Labels

    deferred-needs-inputBlocked on a decision that belongs to the founder

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions