Not sure how to use "expressions"

[iscanegemen]

I'm trying to use the expressions argument of the view instance. Since I couldn't find any documentation for expressions, I don't have enough information about the usage, syntax and the limitations of this feature.

One of the things I want to know is, while creating aggregations via expressions, is it possible to create "stateful" expressions? For example, let's say we want to calculate the average of the last three values t, t-1 and t-2...

Any help would be appreciated, thank you!

[texodus]

Expression documentation is something I am working on right now. The syntax for the language itself is a slightly-modified version of ExprTK, the main difference being double-quoted strings to refer to Columns and single-quoted strings to refer to string-literals, and the use of comment // fields in the first line to rename/alias.

For now, the best way to play with expressions is to use the built-in monaco editor by clicking the New Column button in the column selector. It has (somewhat rough) autocompletion for all built-in functions and your dataset's column names, and will highlight any syntax or type errors in red (though we're working on making this feedback stronger). If you're familiar with JS, you can generate the <perspective-viewer>'s config in the Chrome devtools console once you've found the expression you want like so:

// Copy to clipboard
copy(await document.querySelector("perspective-viewer").save());

I've been working on some examples for this as well, the most complex currently is this Mandelbrot demo which calculates the popular fractal in an expression column and "Heatmap" plugin. The "pixel shader" for this expression looks like this (which you can see in the demo by clicking on the color column's hamburger menu):

// color
var c := float("iterations");
var x := floor("index" / "height");
var y := "index" % "height";

var cx := "xmin" + (("xmax" - "xmin") * x) / ("width" - 1);
var cy := "ymin" + (("ymax" - "ymin") * y) / ("height" - 1);

var vx := 0;
var vy := 0;
var vxx := 0;
var vyy := 0;
var vxy := 0;

for (var ii := 0; ii < float("iterations"); ii += 1) {
    if (vxx + vyy <= float(4)) {
        vxy := vx * vy;
        vxx := vx * vx;
        vyy := vy * vy;
        vx := vxx - vyy + cx;
        vy := vxy + vxy + cy;
        c -= 1;
    }
};

c

1.0.1 does not allow stateful expressions, and I'm not sure yet how/if we will add custom-user stateful functions exactly. However, we will soon support a library of built-in functions that are not row-bound, e.g. window() to grab fixed-size vectors of values around a row in sorted-order (e.g. for moving averages), vlookup() to reference arbitrary rows via index value. I could imagine also potentially adding a last_value() or similar function that returns the previously-calculated value for an expression after update() has been called on a table, which would potentially allow pseudo-stateful user functions that only depend on the last state. I'm open to suggestions for other useful built-ins along these lines, if you have a specific stateful use case in mind, or any ideas on general-purpose user-defined stateful functions.

[jrb3001]

Has the documentation for expressions been completed? We've got perspective integrated into a project and it's hard to tell users what is even possible using them without some reference material. Would also love the non-row bound functions mentioned above; could be very useful. I'd also put in a vote for a set of quantile/ranking functions to make it easy to create multi-factor scores and such.

[brianblakely]

we will soon support a library of built-in functions that are not row-bound, e.g. window() to grab fixed-size vectors of values around a row in sorted-order (e.g. for moving averages), vlookup() to reference arbitrary rows via index value.

Hi @texodus, what is the status of these expression features?

@jrb3001
https://perspective.finos.org/docs/expressions