Extraction layer: start and poll an extraction

[chetanr25]

Extraction layer: start and poll an extraction (#629)

Why

The extraction layer needs a way to kick off an extraction and check on it. This is the first slice of that layer. It gives the frontend a real entry point and a poll point to build against now, before the model side of extraction is ready. The work that turns a narrative into a full contract is a separate, larger piece (#630), so this change keeps the worker as a stub and focuses on the request flow, the states, and the errors.

What this adds

Two endpoints:

• POST /api/v1/extract/{input_id} starts an extraction from an input that was already submitted. It checks the input exists and is ready, makes sure there is not already an extraction for it, and that the model service is reachable. On success it creates the extraction row, queues a job, and returns 202 with the ids and a poll url.
• GET /api/v1/extract/{extract_id} returns the current state. While the job runs it returns the processing shape with a retry hint. Once an extraction is done it returns the completed shape with the full contract, read from the linked incident row rather than from the extraction.

The queued job runs a stub worker for now. It marks the job in flight and leaves the extraction in processing. The real worker that splits the narrative, calls the model, validates, and writes the contract and the draft incident is #630. So while testing this PR an extraction will sit on processing, which is the expected result here.

Errors

Both endpoints return the codes the contract lists:

• input missing, 404 INPUT_NOT_FOUND
• input not ready yet, 409 INPUT_NOT_READY
• extraction already exists for the input, 409 EXTRACTION_EXISTS
• model service down, 503 LLM_UNAVAILABLE
• extraction missing, 404 EXTRACT_NOT_FOUND

One extra fix

While testing I found that sending a body with the wrong content type returned a 500 instead of a clean 422. The cause was in the shared validation error handler. It put the raw request body, which comes through as bytes, into the error response and then could not turn that into JSON. This affected any endpoint that takes a JSON body, not only the new ones. The handler now converts any value it cannot serialize into a safe string, so these cases return 422 as they should. There is a test for it.

Scope and what is next

This is the queue and poll slice only. The real extraction worker is #630. The manual correction endpoint and the readiness and validate endpoints are #631.

A small note for the reviewer that carries into #630 and #631: the completed read validates the stored contract against the incident contract model, so the worker must write a valid contract. Two states that only those later PRs produce, a completed extraction with no incident row and a needs_review status, are not handled by the read yet because nothing in this PR can produce them.

Closes: #629

[marcvergees]

You should have a look at what happens with the linter before merging. There are some linter errors. Lmk if you need help with it.

[chetanr25]

You should have a look at what happens with the linter before merging. There are some linter errors. Lmk if you need help with it.

Very weird, ruff check passes the lint test locally. I will look into what went wrong.

[chetanr25]

High chances that it's related to ruff version difference.

[chetanr25]

There was two issues:

• ruff version was not pinned, hence local and github actions were running two different ruff versions.
• Lint and Tests were configured to run only when a PR was raised to main or development. These test were run only because I had set this PR to be merged to development branch (default branch right now). Now with the last commit fix, these test will run on any PR raised to development-*, this includes branch of approach B and C