Skip to content

feat: Add Australian Bank Sync (Redbark)#1

Closed
oscargws wants to merge 7 commits into
mainfrom
feat/redbark-provider
Closed

feat: Add Australian Bank Sync (Redbark)#1
oscargws wants to merge 7 commits into
mainfrom
feat/redbark-provider

Conversation

@oscargws

@oscargws oscargws commented Jul 25, 2026

Copy link
Copy Markdown
Owner

What

Adds Redbark as a native bank sync provider for Australian accounts. Redbark pulls AU bank data through the official open banking (CDR) framework via Fiskil, an accredited data recipient.

Worth explaining why an aggregator is involved at all: Australian open banking access is a lot more gated than most countries. There's no personal API tier, you need an Australian Pty Ltd company stood up, CDR compliance requirements met, penetration tests, security audits and so on before the banks will hand over data. That's not something a self hoster can do themselves, which is why Australians can't just plug into their banks directly the way Up or SimpleFin users can. Redbark is the accredited middle layer that makes AU bank data reachable with just an API key.

Right now Australians using Sure connect through sure-sync, a docker container I maintain that pulls transactions from Redbark and pushes them into Sure through Sure's REST api on a schedule. It's sitting at 7,000+ downloads with a couple hundred active users, and the most common request I get from them is native support, because nobody wants to run and schedule a second self hosted service just to feed their Sure instance. The container also means hand mapping account ids in env vars, transactions only (no balances or institution metadata), and one more thing to keep updated.

This makes it first class: paste a Redbark api key in settings -> providers and link accounts directly, no extra container to run.

How it works

Same shape as the Lunchflow/SimpleFin integrations:

  • RedbarkItem (encrypted api key, syncable) + RedbarkAccount (encrypted raw payloads), with the usual importer / syncer / provided / unlinking split
  • Provider::Redbark httparty sdk against api.redbark.com/v1, limit/offset pagination
  • Registered via Provider::Factory, wired into family, the providers settings panel, and the account setup flow (Questrade style account type picker, skipped accounts persist an ignored flag)
  • Amounts arrive CDR signed (positive = money in) so they get negated to Sure's convention
  • Scope is banking connections only. Redbark also exposes brokerage connections, those are detected by category and skipped (different endpoint, not a fit for the transactions pipeline)

Credentials

Api key is a deterministic encrypted column (Encryptable), validated on every save. Rotating the key re-arms a requires_update item. Raw provider payloads are encrypted jsonb. The password field renders with value: nil so the key never round-trips to the browser.

Rate limits and failures

  • 429 and 5xx retry with exponential backoff + jitter, capped at 30s
  • Auth failures mark the item requires_update and surface a reconnect prompt
  • Every sync failure goes through DebugLogEntry.capture so it shows in the super admin debug ui, no raw response bodies in logs or errors
  • Balance batch calls exclude stale/non-banking accounts up front and fall back per account, so one bad id can't freeze balances for the rest
  • Account pruning is guarded so a transient empty response can never wipe linked accounts, and nil balances never write a $0 anchor

Reconciliation

No parallel data path. Transactions flow through the existing Account::ProviderImportAdapter (dedupe on redbark_<id> + source), balances through Account::Processor via set_current_balance, merchants through ProviderMerchant. Pending transactions refresh to posted on later syncs.

Migrations

  • create_redbark_items_and_accounts (Migration[7.2], NOT NULLs on lifecycle columns, unique item+account index)

Testing

bin/rails test: 5961 runs, 23886 assertions, 0 failures, 0 errors
redbark tests alone: 52 runs, 90 assertions, 0 failures, 0 errors, 0 skips
rubocop: 2156 files inspected, no offenses detected
brakeman: no warnings
erb_lint: no errors

Also ran end to end against my live Redbark account (mixed banking + brokerage connections). Explicitly tested:

  • Account discovery and linking, including the skip/ignore flow
  • 90 day initial backfill, then incremental sync with the 7 day lookback window
  • Balance updates, including the per-account fallback when a batch is rejected
  • Sign conventions verified against real salary and transfer data
  • Pending -> posted transaction refresh
  • Brokerage connections correctly excluded
  • Key rotation, unlink, and item deletion

@oscargws oscargws changed the title Add Redbark provider integration (Australian bank accounts) add redbark provider integration (australian bank accounts) Jul 25, 2026
@oscargws oscargws changed the title add redbark provider integration (australian bank accounts) feat: Add Australian Bank Sync (Redbark) Jul 25, 2026
oscargws added 4 commits July 25, 2026 18:08
- per family api key provider, built like the lunchflow integration
- syncs accounts, balances and transactions from api.redbark.com
- account setup flow, settings panel, locales and routes
- tests and fixtures
- use DebugLogEntry.capture for sync/import/unlink failures
- retry 429s and 5xxs with backoff, raise on page cap instead of truncating
- keep raw response bodies out of logs and errors
- not null constraints on account columns, migration base 7.2
- persist ignored flag for skipped accounts so they stop nagging setup
- validate api key on every save, re-arm status on key rotation
- destroy aborts if unlink fails, atomic account create and link
- require_admin on mutating actions, see_other on error redirects
- single grouped query for item account counts
- i18n default connection name, blank password field value
- controller and provider tests
- add missing syncable scope, without it every family sync raises
- kick off a sync on connection create and on key rotation
- setup dialog fetches accounts inline for fresh connections and shows api errors
- skip balance write when no balance has been fetched yet, never anchor a false zero
- exclude stale and non banking accounts from the batched balances call, per account fallback if the batch is rejected
- detect the server row ceiling and empty pages instead of silently truncating history
- user sync start date only governs the initial backfill, incremental after that
- fetch connections before the per account loop so auth errors propagate once
- drop untemplated index/show/new/edit routes and dead preload/link_accounts actions
- stable dom id on the settings panel so repeat turbo replaces keep working
- the transactions endpoint 400s for brokerage connections, they belong to /v1/trades
- only import accounts from banking and documents connections
- guard transaction fetches for any legacy linked non banking account
@oscargws
oscargws force-pushed the feat/redbark-provider branch from 7dabb01 to 9673b1c Compare July 25, 2026 08:11
oscargws added 3 commits July 25, 2026 18:38
- treat the truncation header as a pagination signal: split the date window and refetch instead of failing the account
- prune stale pending rows from the snapshot so settled pendings cant come back as duplicates
- block linking a sure account that already has another provider feed
- count setup failures separately from skips and surface an error instead of "all skipped"
- add not nulls on redbark_items name and api key
- enqueue the destroy job after the flag commits, not inside the transaction
- swap bg-gray-400 for bg-surface-inset, drop amounts from info logs, remove i18n default fallbacks
- tests for window splitting, pending pruning and encrypted payload round trip
- benign skips (unlinked account, blank id, unparseable rows) no longer count as failures, tracked separately so a clean batch reports success
- currency parsing goes through extract_currency so hash shaped payloads resolve instead of falling to the default
- merchant ids use truncated sha256 instead of md5
- debug log entries for import failures and account sync scheduling failures
- trim raw_transactions_payload to the current fetch window on merge, same as brex
- keep rows without a parseable date, drop settled pendings as before
- surface skipped rows in the aggregate debug log entry with imported/skipped counts
@oscargws oscargws closed this Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant