Skip to content

anytrellis 0.3.0

Choose a tag to compare

@github-actions github-actions released this 08 Nov 13:19
· 23 commits to main since this release

Added

  • Trellis(page=): a page per level of a column (or columns). A grid (row=, col=)
    shows each level's every cell, in the same place on every page; a wrap shows the level's
    panels, sorted and filtered within it, on several pages with rows= when there are more
    than fit. The pager names the page shown and lists the levels to pick from;
    show_page("north") and page_names do the same from Python. share_x="page" and
    share_y="page" give each page its own scale. With load="pages", a level's rows are
    read as its page is turned to.
  • pairs(): a scatterplot matrix of a table's number columns, x by column and y by row,
    each variable's scale shared along its column and its row. The diagonal shows each
    variable alone (a density curve per colour, or a stacked histogram) over its own range.
    Brushing a panel lights up its rows in every panel; selected() gives them as rows of
    the table, and hovering a point shows its row. corner=True keeps each pair once.
  • Trellis(aspect=): each panel's height as a multiple of its width (1: square).
  • Lazy tables and databases: a Polars LazyFrame, a DuckDB relation or an Ibis table (on
    any database Ibis queries) is read in one query, with only the columns the plot uses,
    when the plot is first drawn. The database computes the panels' summaries: the pages'
    statistics (rows, mean, median, min, max, sd) and order()'s summaries; the trend comes
    from the rows without sorting them. Ten million rows in 1,000 panels build in 3.3 s
    rather than 5.3 s from Ibis on DuckDB (3.2 s rather than 6.6 s from a LazyFrame).
  • Trellis(load="pages"): a paged wrap of a lazy table (DuckDB, Ibis, a LazyFrame) reads
    only the rows of the panels of the page shown and the pages either side of it, and the
    next ones as the pages turn. The panels, their statistics (with the trend), each panel's
    range of x and y, the category levels and the colour ranges come from the database, so
    every page looks as with every row read. Ten million rows in 1,000 panels from DuckDB
    are drawn in 1.0 s from 480,000 rows read (3.9 s reading all of them); a page turned
    reads only the page that came near and makes only its panels' lines and points, keeping
    the others', in 0.2 s. selected() is a query of the selected
    rows; a saved page reads every row of the lines, points and text.
  • Summaries with load="pages": hist, box, violin, density, smooth and
    errorbar of the plot's data are computed by the database, every panel's at once, and
    no row is read: bin counts, quartiles and whiskers, means and spreads, sums of squares,
    and values in fine bins for densities, violins and loess fits (drawn from the bins, to
    within about 0.03%). The axes are every panel's from the start, and pages turn without
    asking again. A brush over a summary selects the rows it stands for, as a query. Ten
    million rows in 1,000 panels: a histogram in 0.8 s (3.2 s reading every row), a loess
    fit in 3.6 s (22 s).
  • Linking with load="pages": link= selects the rows sharing the brushed rows' keys in
    every panel, as a query. anytrellis.link(..., on=) links a plot reading pages to any
    other: its selection is read as the keys chosen, and a selection elsewhere lights up the
    rows of the pages read and fills its summaries to the share of the keys chosen, which
    the database counts in a query per summary (0.2 s for 4,000 of 5,000 keys over ten
    million rows). A link of a plot reading pages needs on=.
  • Paged wraps send only the panels near the page shown: in a notebook, the browser gets
    the data of the page shown and the pages either side of it, and more as the pages turn,
    are sorted or filtered, or a panel of another page opens; a page further away says it
    is loading until its data comes. A thousand panels of a week's readings start with
    2.2 MB instead of 46 MB. A page from save_html or to_html still holds every panel.
  • Level of detail: what goes to the browser grows with the pixels shown, not the rows. A
    panel of more than 20,000 points of a line sends each series thinned (M4: per column,
    its first, last, lowest and highest point), and a panel of more than 100,000 points is
    drawn as cells, each in its points' colours mixed in proportion and more opaque the more
    it holds; hovering one tells its points, of each colour. Zoomed in, in a notebook, Python
    sends the panels shown in more detail: a line at two columns a pixel, finer cells, or
    the points again. Four million points send about 1 MB instead of 128 MB, and their
    save_html page shrinks from 171 MB to 1.4 MB. Selections still use every row;
    save_png and save_svg draw the detail shown; a saved page shows the plot as it
    starts.

Changed

  • A lazy table (a Polars LazyFrame, a DuckDB relation, an Ibis table) is read when the
    plot is first drawn, with only the columns it uses, not collected whole when the plot
    is made. A panel of more than 20,000 points of a line or 100,000 points shows them at
    the level of detail its pixels need (see Added); zoom in for more. No code changes. See
    Upgrading.
  • Faster selections over linked rows (link=): number and category link columns are
    keyed from their values and codes, not their text; and dense points' cells are counted
    and grouped without sorting their points.
  • Faster plots of many rows in panels: rows find their panel by their column's codes, not
    their text, and numbers as panels are labelled once per value, not once per row. Four
    million rows of a line in four panels build in 1.2 s instead of 2.2 s (by a column of
    numbers: 1 s instead of 4.2 s), and in 1,000 panels ordered by a median, with statistics
    to sort pages by, in 3.5 s instead of 93 s.
  • Text columns are encoded by PyArrow when it is installed, not labelled row by row in
    Python: ten million rows of text facets read in 0.15 s instead of 4 s. A Polars
    DataFrame of ten million rows in 1,000 panels builds in 4.2 s instead of 6.1 s.
  • Rects edge to edge (heatmap tiles, cells) show no seams between them in Canvas 2D, the
    browser's image export, save_png and save_svg: their edges fall on whole pixels.