Skip to content

OpenClear 1.8.0

Choose a tag to compare

@chaffed chaffed released this 10 Oct 01:02
· 47 commits to main since this release

An integration release: cores, online banking providers and identity systems can now
connect to OpenClear the way they expect, and others can extend it without changing it.
It brings:

  • SDKs for Python, TypeScript and .NET;
  • an API for a bank's systems (API clients, decision feed, webhooks, idempotency);
  • SAML and SCIM;
  • embedding in online banking;
  • decision and NACHA return files;
  • plugins;
  • a developer sandbox with an integration checklist.

Upgrading from 1.7.0

  • Run alembic upgrade head. The Docker image does this on startup. It adds new tables
    for:

    • API clients;
    • idempotency keys;
    • decision feed cursors;
    • webhooks;
    • plugin choices;
    • SSO identities and SAML assertions;
    • SCIM;
    • embed launches;
    • the decision outbox;
    • conformance results.

    It also adds new columns on sso_connection, tenant, api_client, decision and
    exception_item, and existing decisions are numbered for the decision feed. Every new
    bank table is under row-level security on Postgres.

  • New dependency: python3-saml. Its xmlsec dependency has prebuilt wheels, so no
    system packages are needed. Optional: pip install 'openclear[docs]' for the docs site.

  • Behaviour changes:

    • Existing Admin groups get api_client:manage.
    • The default ACH return reasons get their NACHA codes where none was set.
    • An identity provider no longer attaches a new membership to an existing person who
      isn't already a member in that scope; add them as an admin first (Security fixes,
      below).
    • In the API contract, file downloads are declared as files.
  • Nothing new runs until you turn it on:

    • plugins (OPENCLEAR_PLUGINS);
    • the decision outbox (OPENCLEAR_ENABLE_DECISION_OUTBOX_SCHEDULER);
    • sandbox sign-up (OPENCLEAR_SANDBOX_SIGNUP_ENABLED; never on an installation with
      real banks);
    • embedding (Settings → Embedding);
    • SAML and SCIM (per connection).
  • Downgrading to 1.7.0 is supported: alembic downgrade b9d1f3a5c7e8. Back up first.
    The new tables and columns are dropped, along with the data in them (API clients,
    webhooks, SSO identities, …). The default return reasons keep the codes they were given.

Security fixes (review of this release's new features; SECURITY_REVIEW.md, 2026-10-09)

  • Account takeover by an identity provider: sign-in, token exchange and SCIM no longer
    attach an identity provider's asserted email to an existing person who isn't already a
    member in that connection's scope. An admin adds such a person first. New people are
    provisioned as before.
  • Open redirect: next paths with backslashes or control characters (/\evil.example)
    are refused.
  • Embedding origins: these can no longer inject into the Content-Security-Policy
    header.
  • Sandbox sign-up: it refuses to run on an installation with real banks.

New: developer sandbox and integration checklist (SDK_PLAN.md P4)

  • Developer sandboxes: on an installation run as a sandbox, a vendor gets a free,
    seeded sandbox bank at /ui/sandbox, with its admin sign-in and an API client. It's
    wiped after 30 days. The setting is OPENCLEAR_SANDBOX_SIGNUP_ENABLED; never turn it on
    where real banks are.
  • Integration checklist (Admin → Integration checklist, GET /api/v1/conformance):
    shows what a core or online banking integration has done. A webhook probe checks that
    its receiver rejects a forged signature. See
    docs/guides/certification.md.
  • Test files: scripts/sandbox_test_files.py builds issued-check, X9.37 and NACHA test
    files with Moov, from CheckSim's synthetic checks. expected.csv says what each item
    should become.
  • sandbox/: the sandbox stack. It includes Apache Fineract and the Fineract adapter,
    which was checked against a real Fineract.
  • Docs site: python scripts/build_docs_site.py builds a developer docs site from the
    repo's docs.
  • Docker image: it now keeps decision files and audit archives under /data.
  • Upgrading: run alembic upgrade head. It adds the tenant columns is_sandbox,
    sandbox_expires_at and sandbox_contact_email, and the table conformance_result.

New: the exception.deadline_approaching webhook

  • What it does: sent once per open exception when its decision deadline is near. The
    default is 4 hours before; change it with OPENCLEAR_WEBHOOK_DEADLINE_WARNING_HOURS.
  • When it's checked: by the disposition sweep.
  • Upgrading: run alembic upgrade head. It adds the column
    exception_item.deadline_warned_at.

New: decision files for cores (SDK_PLAN.md P3)

  • Decision outbox: turn it on under Settings → Decision files. OpenClear writes each
    run's new decisions to a file on the bank's schedule and sends the decision_batch.ready
    webhook.
  • Getting the files: a core downloads and acknowledges each one at
    /api/v1/decision-batches.
  • NACHA return files: the new nacha_returns layout writes the ACH debits the bank
    returned as a NACHA return file for its ACH operator. Set the bank's routing number and
    the operator's under Settings → Decision files. The files pass Moov's ACH validator.
  • Return reason codes: the default ACH return reasons now carry their NACHA codes
    (R01, R02, R03, R08, R09, R10, R11). An existing bank's defaults get them too, unless
    the bank set its own.
  • For plugin authors: decision layouts take the bank's settings:
    write(feed, settings), with config_fields.
  • Scheduling: set OPENCLEAR_ENABLE_DECISION_OUTBOX_SCHEDULER=true, or call
    workers.tasks.decision_outbox_job from cron.
  • Upgrading: run alembic upgrade head. It adds the tables decision_outbox and
    decision_batch, and fills in return codes.
  • The openclear command (in the Python SDK): push a file, decisions pull,
    decisions files, and webhooks tail, for scripts and cron.
  • An Apache Fineract adapter (integrations/fineract). It:
    • syncs customers and accounts;
    • sends checks paid from savings accounts as presented items;
    • undoes a returned check's withdrawal in Fineract.
      It's a reference for other cores too.
  • A .NET SDK (sdks/dotnet, OpenClear.Sdk). It's generated from the same contract
    and has the same retries, paging, errors and webhook verification as the Python and
    TypeScript SDKs.

New: OpenClear inside online banking (SDK_PLAN.md P2)

  • Embedding: a bank's business customers can review and decide exceptions inside the
    bank's online banking, without signing in again.
  • How a session starts: the online banking server, acting for the signed-in person,
    asks POST /api/v1/embed/launch for a one-time link (good for a minute). Its page
    shows the link with the new @openclear/embed library (sdks/embed), or in a
    WebView in a mobile app.
  • What the embedded pages are like:
    • no navigation;
    • framed only by the origins the bank lists under Settings → Embedding;
    • they tell the host page when they resize, navigate, record a decision or end.
  • Ordinary sessions still can't be framed at all.
  • Guide: docs/guides/embedding.md. There's also a runnable
    host page in examples/embed-host.
  • Upgrading: run alembic upgrade head. It adds the column
    tenant.embed_allowed_origins and the table embed_launch.

New: online banking sign-on profile and token exchange (SDK_PLAN.md phase F)

  • A connection for an online banking provider: the provider is the identity provider,
    and a customer claim names each person's business customer. Each session is
    limited to that customer.
  • Token exchange (RFC 8693): the provider's server trades a signed-in person's ID token
    for an OpenClear token that acts for them. The token allows only what both the person
    and the API client may do, and the person is created on first use. Link the API client
    to the connection under Admin → API clients.
  • OpenClear-Act-As-Subject: a client can act for someone by the provider's own id
    for them, instead of their email.
  • Upgrading: run alembic upgrade head. It adds the columns
    sso_connection.customer_claim and api_client.sso_connection_id.

New: sign-out from the identity provider, and sign-in from its portal (SDK_PLAN.md phase F)

  • OIDC back-channel logout: when someone signs out at the identity provider, their
    OpenClear sessions end. The provider posts a signed logout token, which is verified
    and accepted only once.
  • Sign-in from a portal tile: a tile in the identity provider's portal (Okta,
    Entra ID) can start OpenClear sign-in through the connection's "initiate login" URL.
    It runs the normal sign-in with PKCE, and only lands on pages on this site.
  • Where to find the URLs: each connection's page lists them, along with the
    redirect URI.

New: SCIM 2.0 provisioning (SDK_PLAN.md phase F)

  • What it does: a bank's directory (Okta, Entra ID, ...) can create, update and
    deactivate people in OpenClear for an SSO connection, and push its groups, at
    /scim/v2.
    • People get access through the connection's existing group mappings.
    • Deactivating someone in the directory, or removing their last mapped group, takes
      their access away and ends their sessions.
    • Nothing is deleted; a deactivated person stays in the action log.
  • Turning it on: on the connection's page. The page shows the bearer token once.
  • The PATCH formats Okta and Entra ID send are both handled, including Entra's
    capitalised operations and "False" strings.
  • Upgrading: run alembic upgrade head. It adds the tables scim_user, scim_group
    and scim_group_member, and the columns sso_connection.scim_token_*.

New: SAML 2.0 sign-on (SDK_PLAN.md phase F)

  • SAML connections: a bank, or one of its business customers, can sign in through a
    SAML 2.0 identity provider (ADFS, Okta, Entra ID, Ping, Shibboleth, ...) as well as OIDC.
    It uses the same group mapping, provisioning and "require SSO" rules.
  • Setting one up: enter the identity provider's entity ID, sign-in URL and signing
    certificate. The connection page then shows OpenClear's entity ID, reply (ACS) URL and
    metadata URL for the identity provider.
  • Security:
    • OpenClear signs its requests with a key pair generated for each connection.
    • Assertions must be signed with SHA-256 or better, addressed to OpenClear, and within
      their validity window.
    • Each assertion is accepted once.
    • An answer must match the request that this browser started.
  • Sign-in started at the identity provider: for example, an OpenClear tile in Okta.
    It's off unless the bank turns it on for the connection.
  • Single logout: a signed logout request from the identity provider ends the person's
    OpenClear sessions.
  • Upgrading:
    • Run alembic upgrade head. It adds new SAML columns to sso_connection, makes
      client_id and client_secret_encrypted optional, and adds the tables sso_identity
      and sso_assertion_seen.
    • New dependency: python3-saml. Its xmlsec dependency has prebuilt wheels, so no
      system packages are needed.

New: plugin matching rules, scoring features and notification channels (PLUGIN_PLAN.md PL3)

  • Matching rules: a plugin can add checks to check and ACH matching, and each bank
    chooses which ones it uses under Admin → Plugins.
    • A rule can only add exceptions, never clear one.
    • A rule that fails flags the item ("A plugin's check failed (review by hand)") rather
      than letting it through.
  • Scoring features: a plugin can add inputs to the fraud model. They apply to every
    bank, because the shared model is trained on all of them.
  • Notification channels: a plugin can send a bank's events somewhere else, such as a
    chat tool. Each bank turns a channel on and fills in its settings; secrets are stored
    encrypted and never shown again. A channel gets the same events as webhooks, retried
    if sending fails.
  • Example and guide: the example plugin has one of each, and
    docs/PLUGINS.md covers them.
  • Upgrading: run alembic upgrade head. It adds the column
    tenant_extension.config_encrypted.

New: Python and TypeScript SDKs (SDK_PLAN.md P1)

  • Two client libraries: sdks/python (openclear-sdk, Python 3.11+, depends only on
    httpx) and sdks/typescript (@openclear/sdk, no runtime dependencies; runs on Node 20+,
    Deno, Bun and Workers). Both are licensed Apache-2.0, so vendors can ship them.
  • Generated from the API contract: each SDK's models and endpoint methods are generated
    from openapi/openclear-v1.json by scripts/generate_sdks.py. A test fails when the
    generated code is out of date.
  • What the hand-written layer adds:
    • client-credentials tokens, renewed automatically;
    • an Idempotency-Key on every write, reused across retries, which cover connection
      failures and responses 429 and 502–504, with Retry-After honoured;
    • page iterators;
    • bulk submit, which splits large lists, follows a 202 to its background import and
      returns results in the original order;
    • file upload;
    • decision feed consumption with acknowledgement;
    • Standard Webhooks verification;
    • acting for a person;
    • typed errors.
  • Tests: both SDKs are tested against a running OpenClear (tests/test_sdk/).
  • The API contract now marks file downloads as files: the CSV, XML and decision-file
    endpoints declare their real content type instead of JSON.

New: plugins (PLUGIN_PLAN.md PL1)

  • What a plugin can add: another payment network, OCR provider, email or SMS
    provider, storage backend, scoring model, or bulk import format. It's a separately
    installed package that declares an openclear.plugins entry point; OpenClear's code
    doesn't change.
  • Turning one on: the operator allows it by name with OPENCLEAR_PLUGINS.
    Installing the package alone activates nothing.
  • When a plugin is skipped: if it targets another plugin API version, fails to load,
    or claims a name that's already taken. The app still starts, and the problem shows
    under Admin → Plugins and in the log.
  • The built-ins work the same way: they now register through the same interface.
    Nothing changes for existing installations.
  • Scoring model: the model is chosen by OPENCLEAR_ML_SCORING_MODEL. It defaults
    to logistic_regression, so this doesn't change unless you set it.

New: plugin import formats and decision files, per bank (PLUGIN_PLAN.md PL2)

  • Plugins can add decision file layouts: the file a core takes back.
    GET /api/v1/decisions/file?layout= returns decisions in any layout the bank can use.
    The built-in openclear_csv is always available.
  • A bank chooses which plugin formats and layouts it uses: a plugin's import formats
    and decision layouts are off for each bank until its admin turns them on under
    Admin → Plugins. The change is audit-logged.
  • Uploading a whole file through the API: POST /api/v1/bulk-uploads (multipart)
    takes a file in any format the bank can use, built in or from a plugin.
    GET /api/v1/bulk-uploads/formats lists them. In the web UI, plugin formats are
    uploaded under Bulk uploads → Other formats.
  • OPENCLEAR_PLUGINS_REQUIRED: OpenClear won't start if a plugin listed here
    doesn't load.
  • For plugin authors:
    • docs/PLUGINS.md explains how to write a plugin, and
      openclear.plugins.api is the stable set of names a plugin imports;
    • examples/openclear-plugin-example is a working template, with a fixed-width
      import format and a fixed-width decision file.
  • Upgrading: run alembic upgrade head. It adds the table tenant_extension.

New: API clients for a bank's systems (SDK_PLAN.md P0)

  • OAuth 2.0 client credentials: a bank admin creates an API client under Admin → API
    clients. It gets a security group (what it may do) and optionally a customer (whose
    data it may see). The system exchanges the client id and secret at
    POST /api/v1/oauth/token for a 30-minute token. Integrations no longer need a
    person's password.
  • A client acts through a service identity:
    • it gets the same permission checks, customer scope, row-level security and audit
      trail as a member;
    • nobody can sign in as it, by password or SSO, and it can't be attached to by email;
    • it isn't listed or edited with the bank's people.
  • Rotating a secret stops the old one at once; revoking a client stops it on its
    next request. GET /api/v1/api-clients lists clients for access reviews.
  • Upgrading: run alembic upgrade head (new table api_client, new column
    user.is_service_account). Existing Admin groups get api_client:manage.

New: a published API contract (SDK_PLAN.md P0)

  • openapi/openclear-v1.json is the API's OpenAPI description, committed, and the base for
    generated SDKs (python scripts/export_openapi.py regenerates it).
  • A test fails on any breaking change to /api/v1: a removed endpoint, a newly required
    input, or a removed response field.
  • The web pages are no longer listed in the API description.

New: customer and account sync (SDK_PLAN.md P0)

  • PUT /api/v1/customers/{customer_number} and PUT /api/v1/accounts/{account_number}
    create or update by the bank's own numbers, so a core can sync nightly without lookups.
    Both have paged lists. Customer-scoped callers only touch their own accounts.

New: webhooks (SDK_PLAN.md P0)

  • Events: exception.created, exception.decided and bulk_upload.completed, sent to
    endpoints a bank sets up under Admin → Webhooks.
  • Standard Webhooks format: signed with HMAC-SHA256, with webhook-id for
    de-duplication. Payloads carry only ids.
  • Delivery: retried for about a day. Each endpoint's page shows recent deliveries, sends
    a test event, replays a failed delivery and rotates the signing secret.
  • Safety:
    • events are written in the same transaction as the change that caused them;
    • URLs must be public https, checked again at connection time, with no redirects
      followed.
    • OPENCLEAR_WEBHOOK_ALLOW_PRIVATE_HOSTS is for single-bank installs with internal
      receivers.
  • New tables webhook_endpoint and webhook_delivery. api_client:manage now covers
    webhooks too.

New: the decision feed for cores (SDK_PLAN.md P0)

  • GET /api/v1/decisions?after=N: every pay or return decision in order, with what a
    core needs to act on it: account, check or trace number, amount, date, outcome, return
    reason and ACH return code. Each decision gets the bank's next sequence number when
    it's made, and the numbers become visible in order, so a consumer never skips one.
  • Also available as a CSV (/feed.csv).
  • A consumer acknowledges what it processed (POST /acknowledge), and
    GET /consumers shows how far behind each one is.
  • Upgrading: alembic upgrade head adds decision.feed_sequence and numbers existing
    decisions per bank in the order they were made (fraud-training examples excluded). It
    also adds decision_feed_cursor.

New: an API client can act for a user (SDK_PLAN.md P0)

  • With the bank's permission per client, a client sends OpenClear-Act-As: <email> to act
    for an existing member: an online banking provider acting for its signed-in customer.
  • The request has only the permissions both hold. The audit log names the person, plus an
    api_client.act_as entry for the client.

New: Idempotency-Key on API writes (SDK_PLAN.md P0)

  • Send Idempotency-Key on a POST, PUT, PATCH or DELETE, and a retry gets the first
    response back (Idempotency-Replayed: true) instead of doing the work twice.
  • The same key with a different request is refused (422). A retry while the first is
    still running gets 409. Server errors aren't kept, so a retry after one runs again.
  • Keys are per caller and kept 24 hours (OPENCLEAR_IDEMPOTENCY_KEY_HOURS); housekeeping
    removes older ones. New table idempotency_record.