Repository navigation
The formula language gets a function registry of 681 built-in functions
with Excel's and Gnumeric's names (0.9.0 had a few scalars): math, text,
dates, statistics, finance, lookups, arrays, LET and LAMBDA (named
LAMBDAs too), engineering, options, distributions, matrix analysis, and
FIXPOINT for iterative models such as PageRank. Every function has
parameter names, help, autocomplete with ghost arguments in the GUI and
TUI, and located error messages.
pg_improv 0.10.0 keeps large models live. Aggregated bindings (sum,
count, min, max, avg and the new count_distinct) and new items
apply incrementally in logical mode: on 2.5 million rows, a 1,000-row
batch is visible in about 0.1 s with no rebuild. There is a new commit
sync mode; you can redefine and drop measures from SQL; dimensionless
results are supported; and named LAMBDAs are kept in improv.lambda.
New here: a DuckDB extension, improv, which loads into DuckDB 1.5.6 and
2.0; Mentat 1.16.0 from crates.io; Verus-proven integer kernels; and every
release is published to PGXN.
Phases P1–P7 of the built-in functions plan: 674 functions (plus
aliases) with Excel's names, typed errors, the IF family,
positional forms and recurrences, lookups, list results, LET and
LAMBDA, and locale-aware entry.
Added
-
Category parameters in named LAMBDAs. A parameter the body uses
where a category goes (OVER c,POSITION(c),RELABEL's categories)
takes a category at the call, soLOGNORM = LAMBDA(x, c, LN1P(x) / LN1P(MAX(x OVER c)))is called asLOGNORM(Authored, Person). A
reduced parameter takes a measure, which keeps the body'sOVERwhen
the call expands. -
Named LAMBDAs in pg_improv.
improv.define(model, 'LOGNORM = LAMBDA(…)')stores a named LAMBDA in the new catalog table
improv.lambda(dumped with the model); the model's formulas call it
by name. Redefining one (replace => true) recomputes every measure
that calls it, in the caller's transaction, writing only the rows whose
value changed; it is an error naming the measure when it would break
one or change its type or dimensions.improv.drop_measuredrops one,
and refuses while a measure or another lambda calls it. The GUI's
Postgres store reads and writes the same table (it creates it on an
older catalog). A model it saved with its lambdas inimprov.meta
still loads, and its next save moves them into the table. -
Redefine and drop measures from SQL.
improv.define(model, definition, replace => true)redefines a derived measure. Re-running
the same definition changes nothing: nomodel_versionbump, no
result row written. A new formula recomputes the measure and its
dependents in the caller's transaction, in every sync mode, and
writes only the rows whose value changed.improv.drop_measure(model, measure)drops a measure: a derived measure's result table, or an
input's binding and cells. A replacement that changes the value type
a dependent reads, and a drop of a referenced measure, are errors
that name the dependent. -
Dimensionless results in pg_improv. A derived measure with no
categories, such asMaxVol = MAX(Vol OVER Person), gets a one-row
result table with just avaluecolumn.recalc, every sync mode,
what_if(itscoordis{}) andstatushandle it;defineno
longer refuses it. -
Role-playing categories:
RELABEL(x, From, To)andSWAP(x, A, B).RELABEL(R[Dst, Iter], Dst, Src)ranges overSrc × Iterwith
the same values;W > 0 AND SWAP(W, Src, Dst) > 0is the reciprocal
test (W read with its two roles exchanged), with no@ITEMor
XLOOKUP. Items match by name; one with no namesake is dropped. See
The Formula Language → Roles. -
COALESCE(a, …, default)andFILL(x, default): a value for a missing
cell.COALESCEhas a cell wherever an argument over all the result's
categories has one, and takes the first argument with a cell there;
FILLhas a cell at every item combination of x's categories. With
Authoredset for 2 of 4 people,COALESCE(Authored, Vol * 0)and
FILL(Authored, 0)both give 4 cells, whereIFNAandIF(ISNUMBER(…))
give 2. Both are an outer union in the dataflow (the arguments' cells,
tagged by position, reduced to the first per coordinate), so input edits
stay incremental;IF's laziness is unchanged. See The Formula
Language → Missing cells. -
FIXPOINT(init, LAMBDA(r, body), [max_iter=100], [tolerance=1e-10]).
A measure that is the fixed point of a step: r = init, then r = body(r),
until Σ|Δ| ≤ tolerance or max_iter rounds pass. Past max_iter it is the
last iterate, never a hang (max_iter ≤ 10,000), so the language stays
total. The engine solves it round by round, warm-starting from the
previous result after an edit. A round evaluates the step's plan in
memory, with the cells of the measures it reads fixed for the solve
(they still update incrementally on the dataflow); a step the host
evaluator does not cover (PREV, recurrences) is a dataflow round. The
two give the same cells bit for bit. The customer's PageRank (damping
0.85, dangling mass spread uniformly) on 1,500 nodes and 100,000 edges
matches a plain power iteration to 3e-15 in L1, in 14 rounds and 0.44 s
(release), and to the same after one edge weight changes (0.07 s). On
10,000 nodes and 1,000,000 edges it takes 6.9 s, 1.2 s after an edit.
See Fixed points in the formula guide. -
Measures reduced together may range over different categories.
They need to share only theOVERcategories. The rest broadcast, and
the result keeps them:SUMPRODUCT(r, M OVER Src)withroverSrc
andMoverSrc × Dstgives one value perDst, andXLOOKUP(k, Key OVER Dst, X)takes anXwith more categories thanKey. These
were "the measures reduced together must range over the same
categories". -
RANK.DENSE(x OVER T, [order])andCOUNTUNIQUE(x OVER T).
RANK.DENSEequals SQL'sdense_rank() OVER (ORDER BY x DESC), ties
included (order 1 ranks ascending);COUNTUNIQUE(Google Sheets' name)
equalscount(DISTINCT x). The goldens are checked against PostgreSQL's
dense_rank()andcount(DISTINCT …); LibreOffice and Gnumeric have
neither function. -
pg_improv: aggregated bindings are incremental in logical mode. A
measure bound withagg(sum,avg,min,max,count,
count_distinct) keeps per-coordinate fold state in the sync worker,
so an INSERT, UPDATE (also one that moves a row to another coordinate)
or DELETE updates its cell without a rebuild; deleting one of two rows
that share acount_distinctkey keeps the count. ATRUNCATEremoves
the relation's cells without a rebuild too. Relations an aggregated
binding reads getREPLICA IDENTITY FULL. A textmin/maxbinding
still rebuilds on change. -
pg_improv: new items without a rebuild in logical mode. A row with
a dimension value the model has no item for no longer rebuilds the
model: the worker mints the item throughimprov.mint_itemsin its
own transaction (same ids and order as a load) and feeds the row to the
live engine. It still rebuilds when a formula reads that category's
item list (POSITION,PREV/NEXT, windows,INTO,FILL,RANK,
…; the README lists them).improv_engine::session::Engine::serves
tells whether a grown model still compiles to the engine's graph. -
pg_improv:
improv.statuscounts the worker's applies and
rebuilds. Two new columns:incremental_applies(batches applied
to the live engine) andrebuilds(full loads). The 0.9.0 → 0.10.0
update script adds them toimprov.sync_stateand recreates
improv.status. -
pg_improv: trigger mode loads only the touched groups. When the
recompute is element-wise, each binding is read under aWHEREon the
statement's touched dimension values, so an aggregated binding loads
only those coordinates'GROUP BYgroups, not the whole fact table.
Commit mode still recomputes the written relations' dependents in full
(the README says why). -
pg_improv: distinct counting.
improv.bind_measure(model, measure, rel, key_col, agg => 'count_distinct')counts each coordinate's distinct
key_colvalues, ascount(DISTINCT key_col)does (the result is a
number). Theimprov.binding.aggCHECK accepts it; the 0.9.0 → 0.10.0
update script widens it. DuckDB'sinput X = col count_distincttoo. -
A DuckDB extension,
improv.improv_calc(formula, inputs)
evaluates any Improv formula on named inputs from a JSON object
(improv_calc('ROUND(PMT(rate/12, n, -pv), 2)', json_object(...))),
NULL for an error value;improv_calc_textreturns any type and renders
errors (#DIV/0!).improv_functions()lists the 674 built-in
functions with signatures (query :=searches them).improv(source, model)reads a table or query, builds a model from a short spec
(category Time = year,input Quantity = qty sum,Revenue = Price * Quantity) and returns one row per computed cell. Errors name the spec
line. It uses DuckDB's stable C API, so one binary loads into DuckDB
v1.5.6 and v2.0. The DuckDB-free logic is a new crate,
improv_duckdb_model. See Improv in DuckDB. -
Formally verified engine kernels. A new crate,
improv_verified,
holds integer kernels with Verus
specifications and machine-checked proofs, and the engine calls them:
GCD/LCM(the result is the greatest common divisor, andLCM's
2⁵³ overflow check is exact), base conversion (DEC2BIN,HEX2DEC,
…,BASE: decoding an encoding gives back the value, every digit is
in range), the 48-bitBIT*operations, the derived-measure build
order (each measure after its dependencies; an error exactly when
there is a dependency cycle),TAKE/DROP's index range, and the
coordinate-key projection and union behind every dimensional join (a
sorted merge, which makes a 1M-cellPrice × Qtyevaluate about 18%
faster).scripts/verify.shand a CI job check the proofs. See
Architecture & Design → Formal verification. -
GUI formula autocomplete. In both formula editors a name being
typed drops a list of matching functions (with their signatures and
descriptions), measures and categories; Up/Down select, Tab or Enter
accepts (a function asNAME(, caret after the(), Esc closes, and
with the list closed Enter commits. The rest of the selected name, or
inside a call the parameters still to give, trail the caret in gray,
and a tooltip shows the signature with the current argument in bold. -
GUI function help. Help ▸ Functions… (or F1, also while editing a
formula) opens a window searching the built-in functions by name,
alias and description, the matches grouped by category with their
signatures, and the selected one's category, aliases, description and
argument types. Insert (or a double-click, or Enter) putsNAME(into
the formula being edited, at the caret. -
GUI located formula errors. Both formula editors (the selected
row and theName = formulaline, which before reported only to the
status bar) underline where a formula fails and show, under it, the
message and a hint: the nearest measure or category names for an
unknown one, what is missing or extra in a call, the(left
open, theOVERform an aggregation needs. A type or dimension error
underlines the first function or measure its message names. -
TUI function help and formula editing.
?(or F1, also while
editing) opens a search over the built-in functions: the matches with
their signatures, and the selected one's category, aliases and
description. Enter insertsNAME(into the edit.e/Enter on a
derived measure edits its formula (symbolic or controlled English),
with a caret. Tab completes function, measure and category names
(again to cycle the matches); inside a call the status line shows its
signature with the current argument marked, and the parameters still
to type trail the caret, dimmed. A parse error keeps the edit open with
the bad token underlined, a^~~~marker under it, the message, and a
hint: "did you mean …?" for a misspelled name, the signature for a
wrong argument count, the(left open. -
pg_improv
commitsync mode (pg_improv 0.10.0;
improv.set_sync_mode(model, 'commit')). Results commit or roll back
with the writer's transaction, as intriggermode, but nothing is
installed on your tables, and the model recomputes once per transaction,
just before COMMIT (or PREPARE), not once per statement: 100 one-row
updates in one transaction on a 10k-cell model commit in 82 ms against
6.6 s in trigger mode. A read before COMMIT sees the old results
(improv.recalc(model)in the transaction reads your writes), and a
recompute that fails aborts the COMMIT with an error naming the model.
Needspg_improvinshared_preload_libraries(or
session_preload_libraries);set_sync_moderefuses it otherwise. As
with trigger mode, the role that enables it needs the TRIGGER privilege
on the bound tables, and every writer must be able to write the result
tables.ALTER EXTENSION pg_improv UPDATEfrom 0.9.0 adds it. -
Distributions (phase P7, 72): Gnumeric's
BERNOULLI,CAUCHY,
EXPPOWDIST,GEOMDIST,LANDAU,LAPLACE,LOGISTIC,PARETO,
RAYLEIGH,RAYLEIGHTAIL,SNORM.DIST.RANGE,OWENT,
CUM_BIV_NORM_DIST; and the R-styleR.D*/R.P*/R.Q*for BETA,
BINOM, CAUCHY, CHISQ, EXP, F, GAMMA, GEOM, GUMBEL, HYPER, LNORM, NBINOM,
NORM, POIS, RAYLEIGH, SNORM (skew-normal), ST (skew-t), T and WEIBULL,
plusR.PTUKEY/R.QTUKEY, with R'slower_tailandlog_pflags. 456
goldens (Gnumeric 1.12.57; mpmath where Gnumeric is wrong: its bivariate
normal is ½ too high for some sign combinations,R.DSNORMis wrong off
location 0,R.PSTignoreslog_p). -
Analysis reductions (phase P7, 30): matrices as a measure over two
categories (MDETERM(A OVER R, C),LOGMDETERM,MINVERSE,
MPSEUDOINVERSE,CHOLESKY,EIGEN,LEVERAGE; results row by row,
INTO R2, C2for a block); time series (HPFILTER,INTERPOLATION,
FOURIER,PERIODOGRAM);SSMEDIAN,CRONBACH,LOGREG,LOGFIT;
the normality testsADTEST,CVMTEST,LKSTEST,SFTESTand their
.STATstatistics;FORECAST.ETS(.ADD),.MULT,.STAT,
.STAT.MULT,.SEASONALITY, a port of LibreOffice's Holt–Winters, and
.CONFINT/.PI.MULTfor seasonality 0. 289 goldens (Gnumeric 1.12,
LibreOffice), most to 1e-12. -
Operands that share a category join over it:
A[R, K] * B[K, C]
spans R × K × C, so a matrix product isAB = A * BthenSUM(AB OVER K)(in place ofMMULT). Measures with no category in common are still
refused. -
Math extras (phase P7, 46):
LAMBERTW,AGM,GD,ILOG,
REDUCEPI,FIB; number theory (ISPRIME,ITHPRIME,PFACTOR,
NT_D,NT_SIGMA,NT_PHI,NT_MU,NT_OMEGA,NT_PI,NT_RADICAL,
PARTITIONS);FLT.MAX,FLT.MIN,FLT.RADIX,FLT.NEXTAFTER; the
movable feasts (EASTERSUNDAY,ASCENSIONTHURSDAY,ASHWEDNESDAY,
GOODFRIDAY,PENTECOSTSUNDAY);DATE2JULIAN,HDATE_JULIANand the
Hebrew calendar (HDATE,HDATE_HEB,HDATE_DAY,HDATE_MONTH,
HDATE_YEAR,DATE2HDATE,DATE2HDATE_HEB); Erlang-B (PROBBLOCK,
OFFTRAF,DIMCIRC,OFFCAP); text (ASC,JIS/DBCS,BAHTTEXT,
ROT13,ENCODEURL,ROMAN,ARABIC). Bounded so each call finishes
in about 2 ms. 303 goldens. -
Engineering functions (phase P7, 72, a new Engineering category):
base conversion (BIN2DEC…OCT2HEX,BASE/NUMTOBASE,
DECIMAL/BASETONUM,HEXREP); 48-bitBITAND/BITOR/BITXOR/
BITLSHIFT/BITRSHIFT,DELTA,GESTEP; 45 complex-number functions
over"a+bi"text, with Gnumeric's inverse trigonometric and hyperbolic
ones,IMFACT,IMGAMMA,IMIGAMMA;BESSELI/J/K/Y;CONVERT
with Excel's unit table and SI/binary prefixes. 530 goldens (LibreOffice,
Gnumeric, Excel's docs). -
Option pricing (phase P7, 32 functions): Gnumeric's
OPT_BSand its
Greeks (_DELTA,_GAMMA,_RHO,_THETA,_VEGA,_CARRYCOST),
Garman–Kohlhagen, French, Merton jump diffusion, Miltersen–Schwartz,
Barone-Adesi–Whaley, Bjerksund–Stensland, Roll–Geske–Whaley, a binomial
tree, executive, forward-start, time-switch, simple and complex chooser,
compound, writer-extendible, two-asset correlation, exchange (European
and American), Kirk's spread, fixed- and floating-strike lookbacks; and
LibreOffice'sOPT_BARRIER,OPT_TOUCH,OPT_PROB_HIT,
OPT_PROB_INMONEY. 200 goldens from Gnumeric 1.12.57 and LibreOffice;
where Gnumeric's bivariate normal is wrong, mpmath. -
More list and LAMBDA forms.
INTO R, Clays a list over several
categories row by row (Excel's 2-D spill);LINEST/LOGESTtake any
number of x's (LINEST(y, x1, x2 OVER T, TRUE, TRUE INTO Row, Col)
gives Excel's 5 × (k + 1) block, checked against LibreOffice);
SORTBYtakes several by/order pairs;SELECTS(values, tests OVER T, condition);SELECTLIST(SELECTwith a lookup over its own
category);MAKEARRAY(R, C, LAMBDA(r, c, body));SEQUENCE(T, [start], [step]);BYROW/BYCOL(x OVER C, LAMBDA(row, SUM(row))). An
expression that reads no measure (positions, literals) now broadcasts
with anything, soPOSITION(R) * 10 + POSITION(C)spans R × C. -
Named LAMBDAs.
improv define model.db 1 "Margin = LAMBDA(p, c, p - c)", thenMargin(Price, Cost)in any formula. Calls are stored as
calls and expanded when compiled, so redefining a LAMBDA changes every
use; a body reads only its parameters and earlier LAMBDAs (never
itself). Saved in the Mentat (:meta/lambdas, installed over older
stores on open) and Postgres (improv.meta) stores; the GUI and CNL
print calls by name. -
LET,LAMBDA,MAP,SCANandREDUCE. LET names values for its
body; a LAMBDA is called on the spot or passed to MAP (cell by cell),
SCAN (a running fold along a category,SCAN(0, x OVER Time, LAMBDA(acc, v, acc + v))) or REDUCE (its last value). Folds compile to
recurrences and update incrementally; a LAMBDA cannot recurse, so every
formula still terminates. -
List results (array functions) land along a category named with
INTO:Top = SORT(Sales OVER Product, -1 INTO Rank)puts the largest
value at Rank's first item:SORT,UNIQUE,FILTER,SORTBY,
TAKE,DROP,MODE.MULT,LINESTandLOGEST(one x; with stats,
Excel's 5×2 block row by row),TEXTSPLIT(each cell its own list, no
OVER needed) andFREQUENCY(bins as separate numbers,
FREQUENCY(x OVER T, 10, 20 INTO Bin)). Excel's empty-array#CALC!
is#N/A. -
Lookups (phase P6), reductions whose key is a parameter (a key over
another category gives one lookup per key cell):XLOOKUP,XMATCH,
MATCH,LOOKUP,INDEX(VALUEAT), Quantrix'sSELECT,
SELECTBETWEEN,SELECTGREATERTHAN,SELECTLESSTHAN; the database
functions as aggregations withWHERE(DSUM,DAVERAGE,DCOUNT,
DCOUNTA,DMAX,DMIN,DPRODUCT,DSTDEV,DSTDEVP,DVAR,
DVARP, andDGET);ARRAYTOTEXT. -
Lookups' shape: an aggregation's parameter may range over categories
the reduced measure lacks; the result gains them and each cell reads the
whole table (TEXTJOIN(Sep, TRUE, x OVER Product)withSepper Time). -
72 statistical reductions (phase P4), each
F(x OVER T, …):- descriptive:
COUNT,COUNTA,COUNTBLANK,SUMA,AVERAGEA,
MAXA,MINA,AVEDEV,MEDIAN,MODE(MODE.SNGL),GEOMEAN,
HARMEAN,TRIMMEAN,DEVSQ,STDEV/STDEVP/STDEVA/STDEVPA,
VAR/VARP/VARA/VARPA(and the.S/.Pspellings),KURT,
KURTP,SKEW,SKEWP,LARGE,SMALL,PERCENTILE(.INC/.EXC),
QUARTILE(.INC/.EXC),PROB,SUBTOTALandAGGREGATE; - conditional, with Excel's criteria (
">5","<>x", wildcards
*/?/~):COUNTIF,COUNTIFS,SUMIF,SUMIFS,AVERAGEIF,
AVERAGEIFS,MAXIFS,MINIFS; - sums of products:
SUMPRODUCT,SUMX2MY2,SUMX2PY2,SUMXMY2,
SERIESSUM,POLYNOMIAL,PERCENTOF; - per item (the result keeps the category):
RANK(RANK.EQ),
RANK.AVG,PERCENTRANK(.INC/.EXC),TREND,GROWTH; - regression and tests:
CORREL(PEARSON),RSQ,COVAR
(COVARIANCE.P),COVARIANCE.S,SLOPE,INTERCEPT,STEYX,
FORECAST(FORECAST.LINEAR), Quantrix'sEXPA,EXPBand
EXPFORECAST,CHITEST,FTEST,TTEST,ZTEST(and their dotted
spellings), andAUTOCORREL(GSL's lag-1 autocorrelation).
- descriptive:
-
Positions, neighbours, running totals and recurrences (the plan's
phase P3).POSITION(T),@ITEM(T),@ID(T),@ITEMCOUNT(T);
PREV(x, T, [n], [default])andNEXT; windowsSUM(x OVER T[FIRST..THIS]),AVERAGE(x OVER T[THIS-2..THIS]); and recurrences, a
measure that reads its own earlier values (Balance = PREV(Balance, Time, 1, Opening) + Deposits), computed as an incremental scan in item order. -
Aggregations grow up: several
OVERcategories (SUM(x OVER Time, Region)),OVER ALL EXCEPT Region,WHEREfilters, parameters (one
value per group, literal or a measure), reductions of several measures at
once, and rows in item order. New:FIRST,LAST,S,TEXTJOIN. -
Date arithmetic in days: a date plus a number is a date, the
difference of two dates a number of days. -
New measures (
improv add-derived/define, the GUI's formula list,
pg_improvdefine) take their type and categories from the engine, so a
text- or date-valued formula is a text or date measure, and an
aggregation's result drops the categories it reduces. -
118 built-in functions, with Excel's names (the plan's phase P1):
- the logical family
IF,IFS,SWITCH,CHOOSE,IFERROR,IFNA,
CLEARERROR,CLEARZEROandBETWEEN, all lazy: an error or a missing
cell in a branchIFdoes not choose never shows; - the error functions
ISERROR,ISERR,ISNA,NA,ERRORand
ERROR.TYPE; - 60 math and trigonometry functions (
POWER,MOD,LN,LOG,
SIN…ACOTH,FACT,COMBIN,GCD,HYPOT, …); - 17 rounding functions (
ROUND(x, digits),ROUNDUP,TRUNC,
CEILING,FLOOR.MATH,MROUND,EVEN, …), decimal-correct
(ROUND(2.675, 2)is 2.68); - 26 text functions (
LEFT,MID,FIND,SEARCHwith wildcards,
SUBSTITUTE,TEXTBEFORE,CONCAT,PROPER,VALUE, …), counting
characters, not bytes.
- the logical family
-
The date family, 66 functions (phase P2):
- information:
ISBLANK,ISNUMBER,ISTEXT,ISNONTEXT,ISLOGICAL,
ISDATE,ISNUMBERORDATE,ISEVEN,ISODD,N,T,TYPE; - text formatting:
TEXTwith Excel format codes,FIXED,DOLLAR,
NUMBERVALUE,VALUETOTEXT; regular expressionsREGEX,REGEXTEST,
REGEXEXTRACT(first match only) andREGEXREPLACE, in the Rust
regexsyntax; - date and time:
DATE,TIME,DATEVALUE,TIMEVALUE, the date and
time parts,WEEKDAY,WEEKNUM,ISOWEEKNUM,DAYS360,DATEDIF,
EDATE,EOMONTH,YEARFRAC,NETWORKDAYS/WORKDAYand their
.INTLforms (no holidays argument yet),NOW,TODAY, LibreOffice's
DAYSINMONTH…YEARS, Gnumeric'sUNIX2DATE/DATE2UNIX, and Numbers'
DUR2*over durations as days.
- information:
-
Statistical functions (the plan's phase P4, scalars): 10 special
functions (GAMMA,GAMMALN,BETA,BETALN,DIGAMMA,IGAMMA,
POCHHAMMER,ERF,ERF.PRECISE,ERFC) and 44 distributions under
Excel's names, with legacy names as aliases where they mean the same
thing (NORMDIST=NORM.DIST) and their own entries where they do not
(NORMSDIST,TDIST,BETADIST,HYPGEOMDIST, …): the beta, binomial,
chi-square, exponential, F, gamma, hypergeometric, log-normal, negative
binomial, normal, Poisson, Student's t and Weibull distributions and
quantiles, plusGAUSS,PHI,FISHER,FISHERINV,CONFIDENCE.NORM,
CONFIDENCE.TandSTANDARDIZE. 326 more golden values
(scripts/goldens/statistical.py). Dotted names whose prefixes are not
functions themselves (NORM.S.DIST,T.DIST.2T) parse. -
More aggregations:
PRODUCT,SUMSQ, and over booleansAND,OR
andXOR;AVERAGE(withAVGas an alias).SUM,MIN,MAXand the
rest also take a list of arguments (SUM(a, b, c)). -
Optional and repeating arguments (
ROUND(x)orROUND(x, 2)), and a
parse error that shows the signature. -
One function registry (
improv_core_model::functions) drives the
parser, the type checker, the engine, the GUI's formula text, the CNL and
a generated Function Reference page in the guide. -
634 golden values, checked through both the scalar evaluator and the
dataflow engine: from LibreOffice 25.8 headless
(scripts/goldens/p1.py), or from the Excel, Gnumeric, Numbers and
Quantrix documentation where LibreOffice lacks a function or disagrees
with Excel. -
49 financial functions (the plan's phase P5 scalars), with Excel's
sign convention (cash paid out is negative):- time value of money:
FV,PV,PMT,IPMT,PPMT,NPER,RATE
(Newton's method, then bisection; #NUM! without a root),CUMIPMT,
CUMPRINC,ISPMT,EFFECT,NOMINAL,PDURATION,RRI,DOLLARDE
andDOLLARFR; - depreciation:
SLN,SYD,DB,DDB,VDB, and the French
AMORDEGRCandAMORLINC; - bonds and securities, on the five day-count bases:
ACCRINT,
ACCRINTM, theCOUP*family (COUPNCD/COUPPCDreturn dates),
DISC,DURATION,MDURATION,INTRATE,ODDFPRICE,ODDFYIELD,
ODDLPRICE,ODDLYIELD,PRICE,PRICEDISC,PRICEMAT,RECEIVED,
TBILLEQ,TBILLPRICE,TBILLYIELD,YIELD,YIELDDISCand
YIELDMAT.
Day counting follows LibreOffice; where LibreOffice departs from Excel's
documented results (DURATIONon actual/actual,PRICE/YIELDwith one
coupon left,EFFECT(0, n),DOLLARDE/DOLLARFRbelow 1,ACCRINT's
calc_method), Improv follows Excel. 457 golden values from LibreOffice
25.8 and Gnumeric 1.12 (scripts/goldens/financial.py; Gnumeric for
ODDFPRICE/ODDFYIELD, which LibreOffice cannot compute).
- time value of money:
-
Cash-flow reductions over an ordered category, in item order:
NPV(rate, cf OVER Time),XNPV(rate, cf, dates OVER Time),
IRR(cf OVER Time, [guess]),XIRR(cf, dates OVER Time, [guess])
(Newton's method, then bisection; #NUM! without a sign change or a root),
MIRR(cf OVER Time, finance_rate, reinvest_rate),
FVSCHEDULE(principal, rate OVER Time), and GNU Oleo's financial
management rate of returnFMRR(cf OVER Time, safe_rate, reinvest_rate, reinvest_min). A period with no cell is skipped, not zero (Excel's rule
for blanks in a range); an error flow is the result. 65 golden values
from LibreOffice 25.8 over a column of flows, and from a transliteration
of Oleo'sdo_fmrr(scripts/goldens/financial_reduce.py). -
improv.raise_errors(pg_improv): raise aLANGUAGE improverror value
instead of returning NULL. -
Operators
^,&and%.^is power with the mathematical
convention: right-associative and tighter than unary minus (2^3^2is
512,-2^2is −4; the spreadsheets give 64 and 4).&joins two values
as text, below+/-and above the comparisons.%divides by 100. -
Formulas in your language. The GUI formula bar and
improv define/
add-derivedread formulas in the formula locale (IMPROV_LOCALE, else
the OS locale); in German: LibreOffice's German names, a decimal comma and
;separators (WENN(Price > 1,5; RUNDEN(Price; 2); 0)). The GUI shows
formulas the same way. Models store the English form.
Changed
-
Every release publishes pg_improv to PGXN, or fails. The PGXN step is
its own workflow (pgxn.yml): called by each tagged release, or run by
hand for a tag that missed PGXN. It checks the tag, Cargo and control
versions agree, uploadsci/pgxn-dist.sh's zip (a re-run of an uploaded
version is a warning, not an error), and waits until api.pgxn.org lists
the version. Missing PGXN secrets now fail the release instead of
skipping it, which is how 0.8.0–0.9.0 never reached PGXN. -
Every function names its parameters in signature help and the
Function Reference, instead of their types: Excel's argument names where
Excel has the function, else LibreOffice's, Gnumeric's or R's
(ROUND(number, [num_digits=0]),SUMIF(range OVER T, criteria, [sum_range]),SORT(array OVER T, [sort_order=1] INTO K)). -
Dimension errors say what and why, by name:
A ranges over [Month] and B over [Region]: they share no category, so no cell has both … Reduce one first, e.g. SUM(B OVER Region), or give them a category in common, instead of[CategoryId(1)] vs [CategoryId(2)]. The same for
function arguments, IF,[…]that leaves a category out, and measures
reduced together. -
Mentat 1.16.0, from crates.io. The storage engine is now the
publishedmentatdbcrate (mentat = { package = "mentatdb", version = "=1.16.0" }), no longer a git dependency on Codeberg; cargo-deny drops
its git source and the per-crate license clarifications. No code
changes; storage speed is unchanged (A/B benchmark:
docs/reviews/2026-10-18-mentat-1.16-upgrade.md). -
Errors are typed values, shown as the spreadsheets show them:
#DIV/0!,#NUM!,#VALUE!,#N/A(plus#NULL!,#NAME?,#REF!
fromERROR). Division by zero is#DIV/0!and a domain error or
overflow#NUM!, where they used to beNaN; an error operand
propagates.LANGUAGE improvreturns NULL for an error value (it used to
returnNaN), or raises it underimprov.raise_errors. -
MIN2/MAX2are now aliases ofMIN/MAX, which take any number of
arguments. -
Comparisons order text, dates and booleans, not only numbers.
-
NOTbinds looser than a comparison, as in SQL:NOT Price > 15is
NOT (Price > 15). It used to parse as(NOT Price) > 15, a type error.
Only a formula comparing aNOTof a boolean measure with<,<=,>
or>=changes meaning (NOT a == bandNOT a <> bgive the same
values either way); write(NOT a) < bto keep the old reading.
Fixed
-
The GUI's Postgres store loads
boolmin/max bindings.storage_pg
builtmax(b)for a boolean column, which Postgres does not have; it
now readsbool_or/bool_and, as pg_model's loader does. -
pg_improv:
agg => 'min'or'max'on aboolcolumn. The load
readmax(value), which Postgres does not have forbool, so every
load of such a binding (in each sync mode, and the logical worker's
rebuild) failed. It readsbool_and(min) andbool_or(max). -
pg_improv: a
defineracing a logical-mode batch. The worker read
model_versionbefore taking the model's lock, so adefinethat
committed while the worker waited could see its new results
overwritten by the old formula's. The version is read again under the
lock. -
Calls that never returned now finish in milliseconds. Property tests
over every built-in function (hegel, each call under a time budget)
found loops whose trip count came from an argument:FACT,COMBIN,
PERMUT,MULTINOMIALof counts near 1e15 (now#NUM!once they
overflow);VDBpast 1e5 periods,CUMIPMT/CUMPRINCpast 1e7,
AMORDEGRConce its write-down rounds to 0 (0) or after 1e6 periods
(#NUM!); the Bessel functions past order 1e7 andOPT_BINOMIALpast
1e4 steps (#NUM!; Gnumeric allows 1e5, which took 11 s);
BINOM.INVand theR.Q*discrete quantiles past 2⁵³ trials; the
quantile bisections (now 65 steps in the floats' order, where
BETA.INVwith huge shapes took 16 s);R.QHYPERof huge
populations;R.PTUKEYwith nmeans near f64::MAX;FMRRwith a safe
rate at or below −100% (#NUM!); and the skew-normal and skew-t far
tails. -
Precision.
FLOOR/CEILINGand their variants snap the quotient to
15 digits as LibreOffice does (CEILING(1.17e-10, 117)is 117, it was
0);PV/FVat small rates;R.PBETA's upper tail at a small x; the
incomplete beta just below x = 1, with one shape below the other's ulp,
or with shapes whose sum overflows;R.PF's upper tail at a small x;
beta quantiles under ε above the median (they were 0); the
hypergeometric mass of populations past 1e6. -
Panics.
HDATEpast Gregorian year 29000,IMIGAMMAof a real
shape past 1e12 (#NUM!) andFORECAST.ETSwith a seasonality near
1e19 (#VALUE!);UNIQUE(…, TRUE)of only repeated values is
#N/A, not an empty list. -
IMIGAMMAoff the real axis. The upper regularized value for large
|z| was computed as 1 − P and cancelled:IMIGAMMA("0.5","30+i",FALSE)
gave 1.2e-5+3.4e-5i (it is 5.0e-15−8.1e-15i), the same as Gnumeric 1.12.
For Re z > 0 past |z| ≈ Re a + |Im a|/2 it now uses Legendre's continued
fraction, so all four forms match mpmath to about 2e-13. The
unregularized lower value after the asymptotic series was the regularized
one (IMIGAMMA("1+i","40",TRUE,FALSE)gave 1, not 0.498−0.155i), and
Γ(0, z) is now E₁(z) rather than#NUM!. -
IMIGAMMAleft of the imaginary axis. For Re z ≤ 0 (where neither
continued fraction applies) it gave#NUM!, as Gnumeric does. All four
forms now follow mpmath'sgammaincon the principal branch:
IMIGAMMA("0","-1",FALSE,FALSE)is −1.895−πi (Γ(0, −1) = −Ei(1) − iπ),
through the lower series (or Legendre's fraction where the series would
cancel) and Γ(a, z) = Γ(a) − γ(a, z). At a = 0, −1, −2, … the upper value
is E₁ and its recurrence, the lower one#NUM!(a pole). -
TUI import-csv that breaks a formula says so. Importing into a
measure a formula uses (givingPricea new dimension, say) left every
derived measure blank with only "imported N cell(s)" in the status. The
status now adds "formulas not recomputed: …" with the engine's error. -
Binomial, beta, F and t with very large parameters. The regularized
incomplete beta behindBINOM.DIST,BETA.DIST,F.DIST,T.DIST,
NEGBINOM.DISTand theR.*kernels lost accuracy once both shapes
passed about 1000, and was wrong from about 10⁸:BINOM.DIST(4e7, 1e8, 0.4, TRUE)gave −0.42 (it is 0.500043), andR.QBINOM(0.3, 1e16, 0.4)was 5.7 standard deviations low. For large shapes it now uses
Loader's saddle-point prefactor and runs the continued fraction to
convergence. -
LAMBERTWnear −1/e. For x within about 700 doubles of −1/e it
returned#NUM!: Halley's iteration bounced between two values
without converging. It now uses the branch-point series there, which
matches mpmath to 1e-14 on both branches. -
RANK,PERCENTRANKandTRENDover an item that is text. That
item is now#VALUE!, as in Excel, LibreOffice and Gnumeric, and the
other items keep their results.PERCENTRANK(x OVER T)used to panic
on such an item, andRANKgave it a rank. -
Saving a partially loaded Mentat model no longer deletes data. A
model fromload_partialsaved through its handle used to retract every
cell outside the loaded closure; the save now only retracts the closure's
missing cells, as the Postgres store already did. -
The TUI edits text, boolean and date cells incrementally, as it does
numbers (throughsession::input_cell, like the GUI); it used to rebuild
the whole engine for every non-number edit. -
Measure[Cat]outside an aggregation is refused when the list leaves
out one of the measure's categories. It used to change only the type the
checker saw:X = Quantity[Product]still produced every Time×Product cell.
Inside an aggregation (SUM(Quantity[Product] OVER Time)) it works as
before.
Security
- duckdb 1.10506.1 (was 1.10505.0) moves its build-time
zip
dependency from 6.0.0 to 9.0.4, fixing RUSTSEC-2026-0367, -0368 and -0369
(SUID bits, symlink targets and permissions applied during ZIP
extraction). It only ever unpacked DuckDB's own archive at build time,
but it no longer needs a waiver.