feat(server): batch and job endpoints — the contract a third-party client drives (#29) - #101
Merged
Merged
Conversation
…ient drives (#29) The API could create a project, version a schema, upload media and run an ingest. It could not use any of it: no way to approve a batch, hand an annotator work, or store a label. This is the set that closes the gap, and it is deliberately the external contract — the official UI gets no private endpoints, so whatever a third-party client can drive, it drives through exactly these sixteen operations. Batch create, delete and membership are deliberately absent. A batch is born from an ingest, and curating one out of an arbitrary subset of assets has no caller until M5's gallery. The lifecycle is here because nothing downstream is reachable without it: an annotation may only be written into a batch that is `in_annotation`. `batch_id` on the ingest launch arrives with it — the debt #28 deferred. The objection was never the feature but that a refusal must not leave a caller holding a 202 pointing at a job row nobody wrote. It does not: `enqueue` resolves the batch in the same transaction that inserts the job, so an unknown batch is a 404 and one past `draft` is a 409, both answered synchronously. Paging lands on the batch asset listing and nowhere else, because that is the one collection that can hold fifty thousand frames and M5's gallery is the caller. It bounds the response, not the read — the kernel has no windowed read, so `total` stays the size of the whole batch and a client pages until it has seen that many items. `docs/api.md` states the cost rather than hiding it. Two kernel additions, both because the surface cannot answer a client's question without them. `JobService.batch` over a promoted public `batch_of`: an `AnnotationJob` records only its task group, so a client holding a job id had no route to the schema version its work is judged against. And `VisionSetError.index` — a typed class-level default, no constructor change — which `AnnotationService` sets on the way out of its per-item loop so a bulk refusal can say which annotation caused it. All-or-nothing means nothing was written, so the position is unrecoverable at the boundary. Two traps. A defaulted discriminator is a lie in the contract: pydantic reads the tag out of the input to pick a variant, so a payload that omits it is refused however the field is declared, while the default emits it as optional. The wire geometry and partition bodies carry none. And `from __future__ import annotations` binds that very name, so a package with an `annotations` module cannot have it — `routes/__init__.py` drops it and says why. No migration: `FORMAT_VERSION` stays 11, `VERSION` stays 0.0.1.dev0, and nothing was added to `ERROR_RULES`.
JArmandoAnaya
added a commit
that referenced
this pull request
Aug 21, 2026
…ient drives (#29) (#101) The API could create a project, version a schema, upload media and run an ingest. It could not use any of it: no way to approve a batch, hand an annotator work, or store a label. This is the set that closes the gap, and it is deliberately the external contract — the official UI gets no private endpoints, so whatever a third-party client can drive, it drives through exactly these sixteen operations. Batch create, delete and membership are deliberately absent. A batch is born from an ingest, and curating one out of an arbitrary subset of assets has no caller until M5's gallery. The lifecycle is here because nothing downstream is reachable without it: an annotation may only be written into a batch that is `in_annotation`. `batch_id` on the ingest launch arrives with it — the debt #28 deferred. The objection was never the feature but that a refusal must not leave a caller holding a 202 pointing at a job row nobody wrote. It does not: `enqueue` resolves the batch in the same transaction that inserts the job, so an unknown batch is a 404 and one past `draft` is a 409, both answered synchronously. Paging lands on the batch asset listing and nowhere else, because that is the one collection that can hold fifty thousand frames and M5's gallery is the caller. It bounds the response, not the read — the kernel has no windowed read, so `total` stays the size of the whole batch and a client pages until it has seen that many items. `docs/api.md` states the cost rather than hiding it. Two kernel additions, both because the surface cannot answer a client's question without them. `JobService.batch` over a promoted public `batch_of`: an `AnnotationJob` records only its task group, so a client holding a job id had no route to the schema version its work is judged against. And `VisionSetError.index` — a typed class-level default, no constructor change — which `AnnotationService` sets on the way out of its per-item loop so a bulk refusal can say which annotation caused it. All-or-nothing means nothing was written, so the position is unrecoverable at the boundary. Two traps. A defaulted discriminator is a lie in the contract: pydantic reads the tag out of the input to pick a variant, so a payload that omits it is refused however the field is declared, while the default emits it as optional. The wire geometry and partition bodies carry none. And `from __future__ import annotations` binds that very name, so a package with an `annotations` module cannot have it — `routes/__init__.py` drops it and says why. No migration: `FORMAT_VERSION` stays 11, `VERSION` stays 0.0.1.dev0, and nothing was added to `ERROR_RULES`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #29.
The API could create a project, version a schema, upload media and run an ingest. It could not
use any of that: no way to approve a batch, hand an annotator work, or store a label. This is
the set that closes the gap, and it is deliberately the external contract — the medical-app
scenario. The official UI gets no private endpoints, so whatever a third-party client can drive,
it drives through exactly these routes.
Sixteen new operations across three route modules;
ROUTERSinroutes/__init__.pyis still theonly wiring edit.
Ledger
FORMAT_VERSIONstays 11,VERSIONstays0.0.1.dev0.ERROR_RULES.openapi.json73 KB → 148 KB, 19 → 35 operations.Scope decided up front
Batch create / delete / add-assets / remove-assets are deliberately not here. A batch is born
from an ingest, and curating one out of an arbitrary subset of assets has no caller until M5's
gallery.
BatchServicestill has the methods. The lifecycle is here because nothing downstreamis reachable without it — an annotation may only be written into a batch that is
in_annotation.Three findings worth carrying forward
1.
batch_idon the ingest launch, the debt #28 deferred here. The objection was never thefeature: it was that a refusal must not leave a caller holding a 202 pointing at a job row nobody
wrote. It does not —
enqueueresolves the batch in the same transaction that inserts the job, soan unknown batch is a 404 and one past
draftis a 409BATCH_NOT_EDITABLE, both answeredsynchronously. Two tests assert the source ends up with zero ingest jobs after each refusal.
2. Paging bounds the response, not the read — and says so.
limit/offsetland on the batchasset listing and nowhere else, because that is the one collection that can hold fifty thousand
frames and M5's gallery (#55) is the caller. The kernel has no windowed read, so
window()slicesa full list;
totalstays the size of the whole batch, so a client pages until it has seentotalitems.
docs/api.mdstates the cost rather than hiding it. When the read starts to cost, the fixis a port method and this contract does not move.
3.
VisionSetError.index, so a bulk refusal says which item. A typed class-level default — noconstructor change, so
test_every_mapped_error_can_be_constructed_with_one_argumentstill holdsand
ERROR_RULES' exact-correspondence test is untouched.AnnotationServicesets it on the wayout of its per-item loop through one
_blaming(index)context manager, so the five refusals in_validatestay ignorant of the loop they are called from;server/errors.pypublishes it asdetail.indexbeside theMediaErrorbranch. It is a kernel fact — "the third annotation you gaveme" is about the call — and it is unrecoverable at the boundary, because all-or-nothing means
nothing was written.
deletekeeps the caller's own position for a repeated id:[a, a, b]blames index 2, not index 1.
Two traps this hit
A defaulted discriminator is a lie in the contract.
type: Literal[GeometryType.BBOX] = ...emits
typeas optional, while pydantic reads the tag out of the input to pick a variant andrefuses a payload that omits it. The wire geometry and partition bodies therefore carry no
default on their discriminator, unlike the domain models they mirror. Verified, then pinned by
test_a_partition_with_no_kind_cannot_pick_a_variant.from __future__ import annotationscollides with a module namedannotations. It binds thatname to a
__future__._Feature, so importing the submodule shadows it and mypy reports theassignment.
routes/__init__.pydrops the future import and says why.Otherwise this is #27's trap firing again, as expected:
AnnotationCreate/AnnotationUpdateandBatchApproveconvert throughto_domain()inside amodel_validator(mode="after"), because apydantic
ValidationErrorraised from a route body is neither aVisionSetErrornor aRequestValidationErrorand would answer 500 toprovenance="model"with nomodel_ref, aconfidence of 2.0, a zero-area box, or a
by_sizeof 0. Andncarriesge=1, sinceJobService.next_pendingrefuses a non-positive count with a bareValueError.Kernel changes — two, both small
JobService.batch(job_id), over a promoted publicbatch_of(uow, job)(was_batch_of).An
AnnotationJobrecords only its task group, so a client holding a job id had no route to theschema version its work is judged against.
JobOut.batch_idis what it exists for.VisionSetError.index, above.The batch asset listing needed no new read: the route projects
{asset_id: (job_id, progress)}off the jobs' own progress maps, which is two existing public reads and no join.
Acceptance
tests/server/test_external_client.pydrives a job start to finish over HTTP alone — project →schema → upload → ingest → poll → approve (
by_size, two jobs) → start → next → submit → re-read →mark → complete job → complete batch → paged listing — using none of the test helpers, so the
whole walk is visible in one function. Every request's status is asserted, not just the final
state. Bulk-submit transactionality is asserted by re-reading after a refusal, not by trusting the
status code.
MCP tools this implies, for #35
list_batches,get_batch,approve_batch,start_batch,complete_batch,list_batch_jobs,list_batch_assets,get_job,get_job_progress,start_job,complete_job,next_pending_assets,set_asset_progress,list_asset_annotations,add_annotations,update_annotations,delete_annotations.Watch this in #30
tests/architecture/test_tracked_file_sizes.pycaps a tracked file at 200 KB andopenapi.jsonis now 148 KB, running ~4.2 KB per operation — roughly 13 more operations ofheadroom. #30 adds dataset, release, export and download-by-hash. Not pre-emptively allowlisted;
this is the warning, not the fix.