OpenClear 1.10.0

A release for running OpenClear in production, and for deciding faster. It brings:

• the real caller's address and per-person rate limits behind Cloudflare, Akamai or a load
  balancer;
• security events (sign-ins, failures, lockouts and more) with pull feeds for a bank's SIEM,
  and optional JSON logs;
• lists that search and export across every row, search everywhere, and clearer navigation;
• Pay and Return buttons, check return reasons, deciding several items at once, and "decide
  now" for a customer's people;
• branded pages for every error, and 503s that load balancers understand;
• long replays and evidence exports as background jobs.

Request context and security events (MASTER_PLAN.md M1)

• The caller's address behind an edge.
  • OPENCLEAR_CLIENT_IP_HEADER and OPENCLEAR_TRUSTED_PROXY_CIDRS read Cloudflare's or
    Akamai's client-IP header, trusted only from the edge's ranges.
  • The address is worked out once per request, and used by rate limits, the action log,
    logs and security events.
• Rate limits per person. Signed-in people and API clients are limited as themselves
  (600 a minute by default); others by address.
• Origin lockdown. An optional secret header from the edge (OPENCLEAR_EDGE_SECRET),
  or a recorded attestation. There's a new control, "Edge network set up".
• Request ids.
  • X-Request-ID on every response: the edge's own id when trusted.
  • It's shown on error pages and recorded in logs and security events.
  • The action log records it, with the caller's address, in each entry's signed metadata.
• JSON logs (OPENCLEAR_LOG_FORMAT=json), with the request's context and an access line
  per request. Outbound HTTP URLs are no longer logged.
• Security events.
  • Recorded: sign-ins, failed sign-ins (with the reason), lockouts, second factors,
    tokens issued and refused, SSO failures, act-as and permission refusals, rate-limit
    refusals (sampled), and security-relevant changes from the action log.
  • Pull feeds: GET /api/v1/security-events, GET /api/v1/audit-log (with signatures),
    and GET /api/v1/platform/security-events.
  • The CLI follows them with openclear events tail.
• Serving.
  • A configurable keep-alive in the Docker image (OPENCLEAR_KEEPALIVE_SECONDS, 75).
  • Year-long caching for versioned static files.
  • A JSON 413 under /api.
  • The sandbox image no longer trusts forwarded headers from any address.

Lists, search and shared components (M2: UX wave 0)

• The main lists search the server, across every row.
  • Issued items, paid items, ACH transactions and exceptions. Until now, the filter box
    searched only the rows on the page, so an item on another page looked as if it wasn't
    there.
  • Search by check or trace number, payee or originator, amount (exact or a range), date
    range and account number.
  • Sort any column across every row.
  • Export what's listed as CSV. Each export is in the action log, and is a security event.
  • Other paginated tables now say "Filter this page only".
• Search everywhere from the navigation: checks, ACH entries and exceptions, among what
  the person may see.
• Navigation by work:
  • Exceptions first, with a count of those needing attention.
  • "Action Log" throughout, where both "Audit Log" and "action log" were used.
• Detail pages:
  • Page titles name the item (e.g. "Check 1234 · $5,000.00 · Exception").
  • "Back" returns to the list you came from, with its filters.
• Empty lists say what to do next, with the button for it.
• Shared components:
  • a confirmation dialog in place of the browser's confirm();
  • toasts for "saved";
  • submit buttons that can't be pressed twice while a form is being sent.
• An accessibility check in the test suite. axe-core in a browser checks ten main pages
  against WCAG 2.1 AA; the two problems it found (pager contrast) are fixed.

Long requests as background jobs (M2: WAF wave B)

• A replay of a file over 1 MB, and an evidence export of more than 32 days, run in
  the background: 202 with a run to poll, then a download. Smaller ones answer at once as
  before. openclear audit export waits for the job.
• Verifying the action log runs in the background, and daily with the control tests.
  The Controls page shows the latest result and its age, instead of walking the whole log
  on every view.

Deciding (M3: UX wave 1)

• Pay and Return are two buttons.
  • Paying needs no reason, and can take an optional note.
  • Returning asks why, from the bank's list for that kind of item.
  • Checks now have a list too: altered or fictitious, not authorized, stop payment,
    duplicate presentment, refer to maker, stale dated, signature irregular. Each has its
    X9.100-188 code, which goes with the decision to the core.
  • Paying a high-risk item still asks first.
• Several at once. Tick items on the exceptions list, then Pay or Return them together,
  with a confirmation of how many and how much. Each is decided, or recommended under dual
  control, exactly as from its own page; anything that can't be done is listed.
• Decide now. A customer's people with checks waiting land on them after signing in,
  and every page shows how many are waiting and the soonest deadline. It's amber within two
  hours and red within 30 minutes. The navigation's Dashboard link still opens the
  dashboard.
• The exception page leads with what differs, the decision and the check image. The
  rest is under "More about this exception". Keys: P to pay, R to return; J and K move
  through the list.
• On phones the list is cards, and the decision buttons are full width and stay in
  reach. The Pay and Return colours pass WCAG AA contrast.
• Fixed: "decide by" times read from SQLite showed hours out in the browser. They lacked
  a time zone, so the browser took them for local time.

Error pages

• Branded pages for every error on a web page. Each has a message that fits it, a way
  back and a reference to quote to support. They replace raw JSON or plain text for:
  • an unknown address (404, with search);
  • a method a page doesn't take (405);
  • an expired form (403, "This form expired… reload the page");
  • a broken address or form (422);
  • a file over the size limit (413, stating the limit);
  • too many requests (429, with Retry-After).
• The API answers JSON for all of them. 413 and 429 were plain text.
• A database that can't be reached is a 503 with Retry-After, not a 500. /health
  checks the database and answers 503 when it doesn't answer.
• Self-contained 502, 503 and 504 pages (static/errors/) for a load balancer or edge
  to show when OpenClear can't answer at all.

Upgrading from 1.9.0

• Run alembic upgrade head. It adds the security_event and check_return_reason tables, and gives every
  existing bank the default check return reasons.
• Behind Cloudflare or Akamai, set the client-IP header and ranges (README, "Reverse proxy /
  WAF deployment"). Until then, the new control fails, and everyone shares one rate-limit
  bucket per edge address, as before.