Skip to content

v0.7.1

Choose a tag to compare

@ptu-gh ptu-gh released this 27 Oct 08:49
· 53 commits to main since this release

v0.7.1 fixes figures that came out wrong, closes two names that -p
printed and every known defect left after v0.7.0, and makes the
refresh after a load faster. Transfers pair across accounts first,
Schwab's bad overnight quotes and its withholding read right, an
account transfer between brokers reaches the matcher, and a bank
account a run no longer lists ends. The held and sold split of a gain
is the same on every refresh, and a refresh no longer stops on DuckDB's
internal error. The Wealth Overview becomes an overview: beside net
worth it charts the asset classes over time and shows a figure or two
from every other dashboard. A refresh whose inputs did not change takes
seconds.

Upgrading

  1. Rebuild with make all. Gold migrations 0150 and 0153 to 0158 apply
    on the first read-write open. A gold file the engine creates is in
    the format the dashboards' driver reads. Until this release runs, do
    not use wealthdb reload without a source, compact or init:
    under DuckDB 2.0 they write a file the dashboards cannot open. From
    this release on, wealthdb compact rewrites such a file, and web refresh refuses to serve one and says so.
  2. Run ./collectors/firstcitizens/firstcitizens load and
    ./collectors/raiffeisen_at/raiffeisen_at load. Each adds a table of
    the accounts every run listed and fills it from the runs already
    kept. A gold load of either source refuses to run before this step,
    and its message says so.
  3. Run ./collectors/refdata/refdata load --force once. It re-reads
    the GLEIF answers already kept; nothing is downloaded.
  4. Reload the changed sources in one call, which runs the enrichment,
    lot, exposure and yield passes once: wealthdb reload schwab ubs firstcitizens raiffeisen_at refdata, plus every source of kind
    fidelity (Fidelity and SVB) and each Plaid source. The Plaid
    adapter re-reads its silver.
  5. wealthdb web refresh, then wealthdb mcp restart. The first
    refresh rebuilds every derived table; the held and sold split of a
    few gain rows moves (realized, unrealized and their sum stay).
  6. make demo, then make demo-web.

Breaking for scripts, config and dashboards

  • kind: "auto" is gone: no detection existed. A config that names it
    fails at load; the setup wizard and validation name the registered
    kinds.
  • Schwab: mark_spike_pct (default 10) and mark_revert_pct (default
    2) on the source set when an overnight mark counts as a bad quote.
  • The Wealth Overview drops "Cash vs positions over time", "Investment
    income by month", "Fees & taxes by month" and "Asset class split",
    and their privacy variants. The cash split over time is Exposure's
    "Asset classes over time", the income the Income dashboard's, the
    fees and taxes Cash Flow's operating outflows. The overview's twin
    also drops "Received income" and "Net spend", which read 100 by
    construction there.
  • The dashboards need gold schema 0154. An older snapshot stops the
    provisioning with "run wealthdb web refresh".
  • The web_transactions view is gone. Its table,
    serving_transactions, and the report_transactions model stay.

Fixes

  • The matcher pairs a transfer across accounts first. Out and back on
    one account pairs only as a reversal: legs that read as out and back,
    by one signature or reversal wording, within the tolerance. A match
    line in the transfer overrides still pairs any two.
  • Schwab: a mark that moves more than 10% from the previous mark and
    comes back within 2% of it at one of the next two observations is a
    bad quote. The observations between read the previous mark, tagged in
    valued_at. A move that stays is never held.
  • Schwab: non-resident withholding, prior-year and foreign tax,
    depositary fees and cash in lieu leave journal and reach income,
    spending and the cash flow. A cash journal between two accounts stays
    a cash move. An instrument's class takes refdata's OpenFIGI type
    first, so a crypto trust reads as a fund.
  • An account transfer between brokers, and a transfer between two
    Fidelity or SVB accounts, reads as transfer_in or transfer_out by
    its signs, so the matcher pairs its two legs. A row whose signs
    disagree stays a journal.
  • A UBS payee cut at the bank's 35-character line takes its remainder
    back from the next line, where that line is the rest of a name.
  • First Citizens and Raiffeisen Austria state each account at every
    run that listed it. An account the bank stops listing ends one and a
    half of its cadences later; a day's ledger close holds the day's end.
  • A gain's held and sold split converts a sold lot's start value at the
    rate of the newest snapshot before the bucket. Before, it took any
    account's rate, and two refreshes of one gold could split a holding
    kept in several accounts differently. A holding's sold
    lots also sum in a fixed order, so report_gains comes out the same
    to the bit on every refresh.
  • The engine turns DuckDB 2.0's caching operator off. On a gold with a
    sizeable history it could throw "dict-surviving cache: global
    dictionary column N changed identity" in the gains phase. The defect
    is DuckDB's (duckdb#27940);
    the setting goes once a DuckDB release fixes it.
  • A failed refresh repeats the engine's own error line and exit status
    and says whether a snapshot still serves. It no longer guesses at a
    busy or unwritable gold. A gold copy in a storage format the
    dashboards' driver cannot read is refused at the snapshot, naming
    wealthdb compact.
  • A security that GLEIF links and a name also matches keeps one link,
    GLEIF's, so its size, sector and country come from one share.
  • A share named on two lines of one size fund takes the heavier line's
    size; it no longer depends on the order of the last load.
  • The three lists of company legal forms are one, so the UBS payee
    rejoin, the bond description reader and the spending fence agree.
  • make test runs a Docker collector's tests on a copy of its
    checkout. On the VM's file share a file's inode number could change
    under memory pressure and stop the suite with "import file mismatch".

Privacy

  • A Plaid account's own name is its nickname, so -p prints it as a
    pseudonym; its display name is the institution with the mask.
  • A cash-flow coverage row whose label is its account's nickname masks
    under -p.

Dashboards

  • The Wealth Overview shows, beside net worth, the stalest source.
    Below a full-width "Net worth over time", "Asset classes over time"
    stacks each day's asset classes as shares of that day's total, from
    the same holdings history; the Asset class picker narrows the mix to
    the chosen classes, which then sum to 100, and a click opens
    Allocation.
  • Under "Over the time range": received income, net spend, the spend
    trend, net cash flow, the savings rate and the realized gain. Under
    "At the range's end, today and ahead": the unrealized gain, the
    returns year to date (TWR and MWR), the income yield, the income
    expected over twelve months and the projected net cash flow. Each is
    its own dashboard's card. The overview's pickers reach it where that
    card has the picker, and its description names the pickers that do
    not. A click opens its dashboard with the pickers both share; a
    year-to-date return opens Returns at its year.
  • The privacy twin shows the same figures as their dashboards' twins
    do. The gains read as a percent of their cost basis, and a click on
    them opens nothing, as Gains has no twin.
  • Allocation's As of day defaults to the last day the holdings history
    reaches, not to today, so a gold not refreshed today shows its latest
    holdings instead of empty cards. Its "Top 100 positions" shows a
    stock's size (large, mid, small), the one Exposure gives the share.
  • Metabase's hourly field sync no longer logs a warning on the list
    columns of the yield tables.

Exposure and reference data

  • An issuer whose legal name is in another script reads as its legal
    name in English, else GLEIF's transliteration. The legal name stays
    in the record. Holdings that name the issuer in Latin now join its
    LEI.
  • The divergence check compares a proxy only on what its own file
    states. A proxy whose file gives no sector or country is no longer
    reported as diverging on it.

Speed

  • A refresh does only the work that changed. When nothing was loaded
    since the last one, it reuses the derived tables and sends Metabase
    nothing, so it takes about three seconds. After a load, a refresh
    takes about a quarter less time: the exposure joins hash, the yield
    writes bind one list per column, and the wait for Metabase polls
    every quarter second.
  • Naming several sources to reload resets and loads each of them,
    then runs the enrichment, lot, exposure and yield passes once. Seven
    sources reload in about 40 percent less time than seven single
    reloads.
  • make all builds the engine and the images together and skips an
    image whose inputs did not change. make test runs the suites
    together.
  • Each pass of a refresh, a load and a reload prints how long it ran,
    and a refresh times its steps outside the materialize too: the config
    read, the engine's key check and open, the container's stop and
    start, the wait for Metabase and provisioning.

Docs

  • fidelity-web, schwab-web and chase say that a personal Plaid account
    reads balances, holdings and transactions through the plaid
    collector, where Plaid covers the institution.
  • First Citizens: an incoming wire's row names no originator on any surface the collector reads.