Document that sum and avg over double precision in dataflows truncate each value to a multiple of 2^-24

[sumaiyaazad]

Version: Materialize v26.25.0 (aa7b919), Docker image.

What happens

In a materialized view, sum and avg over double precision truncate each finite value toward zero to a multiple of 2^-24 (about 6e-8). min and max over the same rows return the original stored float values unchanged. A value smaller than 2^-24 in magnitude becomes 0.

How to reproduce

CREATE TABLE t (g int, v double precision);
INSERT INTO t VALUES (1, 0.1), (2, 26.8), (3, 1e-9), (4, -0.1);
CREATE MATERIALIZED VIEW mv AS
  SELECT g, sum(v) AS s, avg(v) AS a, min(v) AS mn, max(v) AS mx
  FROM t GROUP BY g;
SELECT * FROM mv ORDER BY g;

g	sum	avg	min	max
1	0.09999996423721313	0.09999996423721313	0.1	0.1
2	26.799999952316284	26.799999952316284	26.8	26.8
3	0	0	1e-09	1e-09
4	-0.09999996423721313	-0.09999996423721313	-0.1	-0.1

Each group has one row, so the sum should equal the value exactly. Group 4 shows the rounding is toward zero, not down. A constant query such as SELECT sum(v) FROM (VALUES (0.1::float8)) t(v) returns 0.1, because the planner folds it; only the dataflow path truncates.

Why it happens

src/compute/src/render/reduce.rs adds float sums as fixed-point i128 numbers with FLOAT_SCALE_EXP = 24. The code says the values are "mapped to a fixed precision i128 domain to preserve associativity and commutativity", which makes sense for sums that must also support retractions. The fold_vs_dataflow tests already record the difference between constant folding and dataflow, so this looks intended; the request is only about the user documentation.

What we ask

The float type page (https://materialize.com/docs/sql/types/float/) does not mention this. Please document:

• that sum and avg over floats use fixed point in dataflows;
• that each value is truncated toward zero to a multiple of 2^-24, about 6e-8;
• that numeric avoids this quantization, within its documented precision.

Users comparing a view with a batch system would then know what to expect.

[antiguru]

Draft documentation change up in #39330. Thanks for pointing this out.