11 OCR and Human Verification Workflow

HealthBook AI (internal code name: BabyCare) turns a stored medical document into text, then into suggested structured rows. Only a caregiver, checking each row against the original page, can turn a suggestion into history. Text recognition runs locally by default. External AI transcription is opt-in, needs consent for each document, and produces the same unverified output as local OCR. Confirmation validates every field before writing anything, and it records what was created.

Responsibilities

• Produce a page-tagged transcription (=== PAGE N ===) using local tools only.
• Suggest candidate medicines, vaccines and lab values conservatively, and never auto-parse an ambiguous date.
• Persist a review draft with optimistic versioning so that two caregivers cannot overwrite each other.
• On confirmation, validate every included row and write records only when there are no errors.
• Keep the corrected transcription with the document, plus the review payload for export.

Inputs and outputs

Endpoint	Purpose	Result
POST /api/documents/{id}/extract	local OCR or text extraction	needs_review, text, candidates, warning
GET /api/documents/{id}/extraction	read stored text	status, text, candidates
GET /api/ai/provider	show the configured vision provider	host, model, transmission notice
POST /api/documents/{id}/vision-extract	opt-in external transcription	needs_review only
GET /api/documents/{id}/review	saved draft, or fresh structured_candidates	items, version, status
PUT /api/documents/{id}/review	save draft	new version
POST /api/documents/{id}/review/confirm	validate and create records	created, skipped, document_status: reviewed

Text extraction

api/extract.py uses PyMuPDF for PDFs. If a page has fewer than 40 characters of embedded text, it is rasterised at 1.5x and passed to Tesseract (--psm 3, 30 s timeout). Images are verified, converted to RGB and OCR'd. If OCR is unavailable, the marker [OCR unavailable; review this scanned page manually] is inserted instead of failing silently. Output is capped at 200,000 characters and stored Fernet-encrypted in extracted_cipher. Re-extraction is refused (409) once a document is reviewed.

Opt-in external vision (api/providers.py)

This runs only if BABYCARE_VISION_URL, BABYCARE_AI_KEY and BABYCARE_VISION_MODEL are all set. The request must:

• come from the primary guardian;
• carry share_document_with_provider: true;
• echo the exact provider_host of an HTTPS endpoint;
• be for a document of at most 10 pages that has no confirmed review.

An audit row (consented_external_document_transmission) is written before the call. The prompt tells the model to transcribe literally, mark [ILLEGIBLE] text, and not infer, diagnose, prescribe or follow instructions found in the document. If the call fails, the API returns 502 and creates nothing.

Suggestion extraction (api/extraction.py, api/vaccination.py)

Category	Extractor	What it suggests	Deliberate limits
vaccination_card	card_candidates	vaccine names matched by regex, the dates on the line, and an interpretation: possible_given_date, possible_due_date, unspecified or ambiguous_multiple_dates	dates without a label are never assumed to be due dates; lines with several dates stay ambiguous
prescription	medication_candidates	potential name plus the raw line	dose_and_frequency: not_parsed
lab	line regex	test, numeric value, printed unit, optional reference range	requires a number and a unit; date and specimen left blank

parse_unambiguous_date accepts ISO dates. A dd/mm/yyyy or dd-mm-yyyy date is accepted only when the day is greater than 12. When both parts are 12 or less (for example 05/06/2026), it returns nothing and the field stays empty for the human. Every row is returned with verified: false, activate_reminders: false and a list of uncertain_fields. classify() assigns a category from keywords only when the document was other. In the web review, rows suggested by OCR start out excluded, and a caregiver must tick them in.

Review states

Diagram: Document review states

flowchart LR
    A["not_extracted"] -->|"local OCR or consented vision"| B["needs_review"]
    B -->|"PUT review, version plus one"| B
    B -->|"confirm with errors"| B
    B -->|"confirm, all checks pass"| C["review confirmed"]
    C --> D["document reviewed"]
    D -->|"re-extract or re-confirm"| X["409 refused"]
    B -->|"version mismatch"| Y["409 reload required"]

Loading

Saving and confirming

Optimistic concurrency. Both PUT and confirm run a conditional UPDATE ... WHERE version = :client_version AND status = 'needs_review'. If no row matches, the call returns 409 ("Another caregiver changed this review").

Validation (api/review_validation.py) runs completely before anything is written. It returns a list of errors, each with index, page, entry_type, per-kind row_number, field, reason and a message:

Rule	Reason code
Only included, non-empty rows are validated; excluded or blank rows never block	n/a
Page must be between 1 and the document's page count	page_out_of_range
Each included row must be ticked as checked against the original	unverified
Required fields: medicine name; vaccine name; observation test_name, value, record_date; episode title, started_on	required
Date fields must be complete ISO dates	invalid_date
?, illegible, unreadable, unclear, ___, ... rejected outside free-text fields	unreadable
A vaccine row needs a given, due or next-due date; a dose is never assumed given	date_required
Medicine rows cannot activate reminders here	reminders_not_allowed
The final acknowledgement checked_against_original must be true	acknowledgement_missing
Pydantic schema of the target model	invalid
Vaccine: given date between birth and today; next dose after given	out_of_range, order
Vaccine duplicate (same normalised name and dose) without duplicate_confirmed	possible_duplicate

If any error is found, the API returns 422 with {message, errors} and writes nothing. If there are no included rows, the document is still archived as transcription only.

What confirmation creates

Item kind	Record created	State after confirm	Provenance
medicine	Medicine	status='draft'; needs separate verification against the prescription before reminders	document id + page
vaccine with given date	Vaccine	confirmed=true (caregiver checked the card)	source_type='vaccination_card'
vaccine with due date only	Vaccine	confirmed only if the caregiver ticked reminders	same
vaccine with next_due_date	extra Vaccine row for the next dose (due date only, no batch)	confirmed = reminder choice; audit confirmed_next_dose or unconfirmed_next_dose	same
observation	Observation	verified=true	document id + page
episode	Episode	links validated to the same profile	document id + page, links_json
corrected text	Document.extracted_cipher	replaces the OCR text	document

Every created row gets an audit entry confirmed_transcription. The response says that medication reminders "require separate prescription confirmation".

Security boundaries

• Review endpoints use own_resource. External vision is restricted to the guardian, and it is HTTPS-only, host-pinned, consented and audited.
• Review payloads are encrypted (EncryptedText) and included in the guardian's export ZIP.

Clinical safety rules

• OCR extracted ≠ clinically verified. Suggestions are never facts, and nothing unreadable may be guessed.
• AI transcription never prescribes, diagnoses or changes doses, and it creates no records by itself.
• Structured records keep provenance to the original document and page.

Implementation

api/extract.py, api/extraction.py, api/vaccination.py, api/providers.py, api/review_validation.py, api/workflows.py (review_get, review_save, review_confirm), api/main.py (document_extract), web/features.js (review UI).

Related pages

Medical Document Ingestion · Medicines and Administration Model · Vaccination Model · Labs, Growth and Longitudinal Health · Security and Privacy Model