A privacy-conscious, vision→reasoning pipeline that turns a photo of a timecard into a warm, numerically-grounded financial audit, with explicit data boundaries for local and hosted use.
Snap a picture of a timecard. MoneyGuard reads the hours with a vision model, computes your real weekly position locally, reduces the ledger to selected aggregate metrics, and streams back a mentor-style audit from a text model — token by token, rate-limit-safe.
# Create the required private ledger, then run with zero API keys:
cp finance.example.json finance.json
npx moneyguard --mock fixtures/timecard.png📊 Wage Audit (2026-W26)
---
🕒 Labor: 38 hrs
💰 Gross: $950.00 AUD
📉 Burn: $552.38 AUD
💎 Surplus: $397.62 AUD | STABLE
---
🧠 Audit:
先停下来给自己一个肯定——一周四十多个小时的体力活扛下来，
还能自费把学习和 AI 工具一个个续上，这份狠劲本身就值钱…
CLI/library. finance.json is read into memory on the machine running MoneyGuard and is never transmitted. Its line items, labels, and per-item tags never leave that machine. In live mode, the configured vision provider receives the timecard image, while the audit provider receives selected OCR values, aggregate financial metrics, and fixed directives.
Hosted /extract server. This has a different privacy posture. The uploaded image and the extracted hourly rate travel over the network by design: the server sends the image to the configured vision provider and returns hourlyRate to the authenticated web client as part of the documented response contract. Do not treat the hosted endpoint as a local-only workflow.
Hosted /v1/explain server. This additive endpoint accepts only the frozen masked-metrics/topic/profile contract and never reads finance.json. Pipeline keeps request, prompt, response, and stream content in memory only for the request lifecycle. DeepSeek is not approved for real-user production data: the only built-in DeepSeek mode is explicitly restricted to synthetic evaluation, and the endpoint otherwise fails closed until a named provider completes the Provider Privacy Gate.
Someone self-funding a career change into tech needs honest financial feedback, but the input is a photo and the data is deeply personal — exact hourly rate, rent, every subscription. For the CLI/library workflow, two hard requirements fall out immediately:
- The ledger file and its line-item details must remain on the machine running the pipeline. Not in a prompt, not in a log.
- The model must still reason about the real numbers.
MoneyGuard resolves that tension with a local data-minimization boundary: ledger math happens in the local process, and only selected OCR values, aggregated metrics, and fixed tone directives are sent to the reasoning model. This is not a claim that the audit prompt is anonymous: it intentionally retains both hours worked and gross income while that product decision remains open.
flowchart LR
IMG[📷 Timecard image] --> VIS[Vision Provider<br/>Gemini — OCR only]
LED[(finance.json<br/>local ledger)] -.in-memory.-> MET
VIS -->|untrusted JSON| ZOD{Zod validate}
ZOD -->|OcrResult| MET[computeMetrics<br/>pure math]
MET --> PAY[buildAuditPayload<br/>🔒 minimize]
subgraph LOCAL [" 🖥️ Local ledger processing — line-item details stay local "]
LED
MET
PAY
end
PAY -->|selected OCR values + aggregate financial metrics + fixed directives| AUD[Audit Provider<br/>DeepSeek — streamed]
AUD -->|tokens| RPT[buildReport<br/>deterministic skeleton]
RPT -->|onReportUpdate| TX[(Transport<br/>CLI · Telegram)]
Split-brain compute. Two models, two jobs: a vision model receives the timecard image and does structured OCR, while a text model receives the selected OCR values and aggregate ledger metrics needed for empathetic copywriting. Neither is trusted to do the other's job.
The pipeline is the product; transports are disposable. runMoneyGuardPipeline(imageBuffer, { onReportUpdate }) is completely channel-agnostic. The CLI and the Telegram adapter are each ~40 lines that own only transport concerns.
For CLI/library use, the ledger is read in-memory and reduced to tag-aggregated weekly sums and a health tier. The finance.json file, line-item records, labels, and per-item tags are not transmitted. The audit payload intentionally includes selected values such as hours worked and weekly gross income, so it should not be described as anonymous or as hiding an hourly rate that can be inferred from those two values. currentRole is mapped to a fixed coaching directive instead of being interpolated into the prompt, and the reversible hoursFor150 metric is omitted. Tests seed private markers and assert that the raw ledger fields and role string do not appear in the outbound prompt (src/pipeline.test.ts). The hosted /extract posture is separate: its image upload and hourlyRate response cross the network by contract.
You cannot wrap a live token stream in a naive "retry on failure": re-running it replays tokens the user already saw. MoneyGuard uses streamWithConnectRetry, which retries only while the stream fails before its first chunk (connection establishment). Once a single token is emitted, it never retries. Non-streaming calls (OCR) use ordinary withRetry with exponential backoff + equal jitter. Two retry strategies, deliberately not interchangeable — enforced by tests.
Telegram returns HTTP 429 if you edit a message too fast. Every transport gates streamed re-renders to one per 1000ms while always applying the final frame, and keeps a trailing ▌ cursor until the last token lands. Crucially this lives in the transport, not the pipeline — the same THROTTLE_MS = 1000 contract appears in both src/cli/main.ts and examples/telegram/adapter.ts, proving it's a transport property.
OCR output is hostile input. It's parsed with Zod: hours are coerced from possible strings, constrained to (0, 168], and a missing confidence defaults to low while an unexpected confidence is rejected. A blurry photo yields a clean "Vision Error", never a crash.
finance.json is a list of tagged line items with a cadence (monthly/weekly). Aggregation filters by tag and normalizes each item to a weekly amount, without a hardcoded if/else chain. Three of the six accepted tags — essential, strategic_weapon, and discretionary — currently have their own weekly subtotals. The other three — liability, subscription, and variable — are accepted and validated but are not separately aggregated. Adding a category with its own subtotal requires adding the enum value, computing the subtotal in src/metrics.ts, and adding its output line in src/payload.ts: no new branching is required, but it is not a one-line change.
The pipeline depends on two interfaces — VisionProvider and AuditProvider — not on any SDK. That yields three implementations behind one seam: Gemini, DeepSeek, and a deterministic Mock. The mock is what lets the whole pipeline run (and be tested) with zero keys and zero network. Tests inject stubs directly — no module mocking required.
The pipeline returns { ok: true } | { ok: false, kind: "config" | "vision" | "model", message }. Transports never inspect internals — they render message on failure and stream on success. Local config errors (bad JSON / failed schema) are deliberately distinguished from network/vision errors.
Requirements: Node ≥ 22.
git clone https://github.com/liuyuelintop/moneyguard-pipeline.git && cd moneyguard-pipeline
pnpm install # or npm install
# 1) Create your private ledger, then run with zero keys:
cp finance.example.json finance.json
pnpm moneyguard --mock fixtures/timecard.png
# 2) Run the test suite (no keys needed):
pnpm test
# 3) Build the distributable library + CLI:
pnpm buildcp .env.example .env # then fill in your keys
cp finance.example.json finance.json # your private ledger (gitignored)
# .env: GEMINI_API_KEY=... DEEPSEEK_API_KEY=...
pnpm moneyguard path/to/real-timecard.png| Variable | Purpose | Default |
|---|---|---|
GEMINI_API_KEY |
Vision / OCR auth | — (required, live) |
DEEPSEEK_API_KEY |
Audit / text auth | — (required, live) |
MONEY_GUARD_VISION_MODEL |
OCR model | gemini-2.5-flash |
MONEY_GUARD_TEXT_MODEL |
Audit model | deepseek-v4-flash |
MONEYGUARD_EXPLAIN_PROVIDER |
Hosted explanation mode: mock or synthetic-only deepseek-synthetic-evaluation; unset fails closed |
— |
MONEYGUARD_MOCK |
Force offline providers (--mock) |
off |
MONEYGUARD_PROVIDER_MAX_ATTEMPTS |
Bound vision-provider calls, including the initial attempt (1-3) |
3 |
MONEY_GUARD_DEBUG |
Enable safe diagnostics without payloads, headers, secrets, or env values | off |
The private OCR endpoint is started with:
pnpm build
node dist/http/server.jsRuntime contract:
| Setting | Contract |
|---|---|
PORT |
HTTP server listens on this value when provided; otherwise falls back to 10000. |
| Bind host | The HTTP server binds to 0.0.0.0 so Render can route traffic to it. |
HOST |
Ignored by the executable HTTP entrypoint. Use code-level startExtractServer({ host }) only for embedded tests/tools. |
MONEYGUARD_PIPELINE_CREDENTIAL |
Required bearer credential for POST /extract and POST /v1/explain; keep present and masked in hosting UI. |
MONEYGUARD_EXPLAIN_PROVIDER |
Explanation mode. mock is offline; deepseek-synthetic-evaluation is synthetic-only. No real-user production value exists until a provider passes the privacy gate. |
MONEYGUARD_PROVIDER_MAX_ATTEMPTS |
Server-only vision-provider attempt cap from 1 to 3; use 1 for protected single-attempt rehearsal. Missing or invalid values fall back to 3. |
MONEYGUARD_REQUIRE_SINGLE_PROVIDER_ATTEMPT |
Server-only protected rehearsal invariant. When true, MONEYGUARD_PROVIDER_MAX_ATTEMPTS must be explicitly and exactly 1; otherwise extraction fails before provider work. |
GEMINI_API_KEY |
Required only for live OCR provider calls; keep present and masked in hosting UI. |
MONEYGUARD_MOCK |
Must be false or unset for live OCR; set only for deterministic offline tests. |
MONEY_GUARD_DEBUG |
Must be false or unset in hosted rehearsal/production; debug output is not needed for private OCR smoke checks. |
MONEY_GUARD_VISION_MODEL |
Optional OCR model override; defaults to gemini-2.5-flash. |
NODE_VERSION |
Use Node 22 on hosts that require an explicit runtime version. |
finance.json |
Required at process root for hosted totals math; provide it as a host secret file, not a committed file. Unknown string context.marketCondition values are normalized to neutral with the fixed diagnostic market_condition_normalized. |
Every real CLI, library, and hosted extraction run requires an explicit, readable, valid finance.json ledger (or a MoneyGuardConfig.financePath override for library embedding). Ledger validation finishes before any vision-provider call. The bundled finance.example.json is only a template: MoneyGuard never selects it automatically.
POST /extract accepts multipart/form-data with mode=real-ocr and an image file. Authentication is checked before the request body is read. The image cap is 5 MiB, and the total HTTP request cap is 5 MiB + 256 KiB to allow multipart overhead; requests over the total cap return 413 before multipart parsing.
The web adapter may send a UUID v4 in X-MoneyGuard-Correlation-Id. The endpoint validates the fixed 36-character format before logging it, treats missing or invalid values as safe categories without reproducing raw input, and never uses correlation metadata for authentication, provider selection, retry policy, or extraction logic.
Accepted image MIME types are image/png and image/jpeg; image/jpg is normalized to canonical image/jpeg. image/webp is not accepted for the rehearsal contract. The endpoint verifies that the declared MIME type matches a bounded container-structure check before it calls the vision provider, and it passes the validated canonical MIME type to Gemini. The checks require PNG IDAT before IEND and JPEG segment structure with EOI; they are not complete image decoding. Mismatched, malformed, or unsupported image types return 415.
Successful responses are totals-only:
{
"source": "real-ocr",
"extraction": {
"totalHours": 38,
"hourlyRate": 25,
"grossWage": 950,
"currency": "AUD",
"confidence": 0.9,
"warnings": []
}
}The endpoint must never return raw image bytes, raw OCR text, filenames, MIME metadata, worker/employer metadata, or shift rows.
POST /v1/explain is additive and runs in the same hosted process as /extract. Its canonical Phase 1 wire contract is contracts/explain-v1.openapi.json; the accepted decision and companion controls are versioned in ADR-001, the contract freeze, the Provider Privacy Gate, and the synthetic evaluation matrix.
The endpoint authenticates before reading the body, accepts strict application/json up to 8,192 UTF-8 bytes, and accepts only profile moneyguard-mentor-en-AU-v1 with topics risk-signal, surplus, weekly-expenses, or labour-value. It emits normalized text/event-stream events:
event: delta
data: {"text":"..."}
event: done
data: {"finishReason":"stop"}
Failures before the first delta use generic JSON HTTP errors. Failures after streaming begins use one terminal SSE error event. Pipeline performs at most two connection attempts, retries only before the first delta, aborts provider work on client disconnect, and never retries, replays, or switches provider after visible output. DeepSeek requests explicitly disable thinking and set max_tokens: 800; decoded output is capped at 16 KiB.
The deterministic mock path and the executable 48-case fixture at fixtures/explain/synthetic-cases.ts require no provider credential. To exercise that path in a private local server:
MONEYGUARD_PIPELINE_CREDENTIAL=local-test-only \
MONEYGUARD_EXPLAIN_PROVIDER=mock \
node dist/http/server.jsLive DeepSeek evaluation, if deliberately enabled with MONEYGUARD_EXPLAIN_PROVIDER=deepseek-synthetic-evaluation, accepts only exact requests from those 48 versioned fixtures. Any unlisted or altered schema-valid request is rejected as a generic 400 invalid_payload before a provider is constructed or called. This repository does not provide a production real-user mode while the privacy gate remains not passed.
import { runMoneyGuardPipeline, mockProviders } from "moneyguard";
const result = await runMoneyGuardPipeline(imageBuffer, {
providers: mockProviders(), // or omit for env-selected live providers
onReportUpdate: async (text, final) => {
// YOU own throttling/transport here. final === true is the last frame.
process.stdout.write(text + "\n");
},
});
if (!result.ok) console.error(result.kind, result.message);Wiring it to a real Telegram bot is a few lines — see examples/telegram/.
src/
pipeline.ts Orchestrator (channel-agnostic, DI providers, discriminated result)
finance.ts Fail-closed ledger loading and schema validation
metrics.ts Pure finance math — cadence normalization, tag subtotals, tier 🔒
payload.ts Local data minimization → selected upstream audit payload 🔒
resilience.ts Backoff/jitter retry + stream-safe connect-retry + error mapping 🔒
schemas.ts Zod contracts for the ledger and untrusted OCR output
prompts.ts Static persona + OCR prompts (dynamic data is built separately)
report.ts Deterministic Markdown skeleton (model can't hijack structure)
config.ts Single source of truth for env-driven config
providers/ VisionProvider/AuditProvider interfaces + Gemini, DeepSeek, Mock
http/explain.ts Privacy-gated normalized-SSE explanation endpoint
cli/main.ts CLI transport (owns the 1000ms throttle)
examples/telegram/ The original bot transport this was extracted from
🔒 = privacy/resilience redline modules.
pnpm test # full offline suite: math, retry semantics, privacy boundaries, and pipeline paths
pnpm typecheck # strict TS, no implicit any, noUncheckedIndexedAccessThe default suite runs entirely offline via the mock providers. It asserts that ledger records and raw context strings stay out of audit prompts, /v1/explain never reads the ledger, a live stream is never replayed on retry, and all 48 fixed synthetic explanation cases satisfy deterministic contract and privacy checks.
Issues and PRs welcome — see CONTRIBUTING.md. The three architectural redlines (privacy boundary, stream-safe retry, transport throttle) are intentional; please don't regress them.
MIT © 2026 Yuelin Liu