Repository navigation
v0.9.11
Release v0.9.11
📦 NPM packages
🐳 Docker image
docker pull ms2data/malloy-publisher:0.9.11
docker pull ms2data/malloy-publisher:latest
ℹ️ Details
- Release branch:
release/v0.9.11 - Triggered by: @kylenesbit
The builder shows the filters a report reads through its model; the skills put a reader's filters in the report
The builder's filter row now matches the reader's. A report whose tiles read a given through the model (a source's own where: Manufacturer ~ $manufacturer_filter) showed that filter in the reader and not in the builder, which drew only givens bound in the report file. The builder now draws it too, after the report's own filters, in the manifest's order. It can be set, as in the reader, but has no remove badge and no settings: its popover says where it is set and is changed: the model, or a where: on this report's own source: that applies it to every tile there. Any filter's popover in the builder, a fixed one included, can open the report with its current value, written as the report's starting value (## artifact { givens { … } }), or put it back to its default; and a filter a model sets offers Open the model when the host passes ReportEditor's onOpenModel(modelPath) (the Console opens the model's page). previewGivens takes the given names read through the model as a third argument (givensReadThroughModel computes them), and FilterEditState gains fixed, the reason a filter cannot be changed here.
The skills now say where a filter belongs. A filter a reader picks on a dashboard or notebook (a manufacturer, a date range, a "major only" toggle) is declared in the report and bound on its tiles, and the model's sources are left unfiltered. A given stays in the model only when the data's meaning or access depends on it: a gate, a value every query must supply, a parameter a field is defined with, or scoping meant for every consumer. malloy-model, malloy-reports (with a before-and-after worked example), malloy-analysis-report and malloy-define are updated, and an agent that used to put such a filter in the source now puts it in the report.
A required filter asks for a value instead of failing every tile
A given with no default is now required: true on every given the API publishes (Given.required, on the model surface and in a report's manifest). A query that reads one fails until a value is supplied, and a report used to run every tile straight into the compiler's "Given 'RECALL_YEAR' has no value and no default…". Now the reader and the builder run no tile that reads an unset required filter: each such tile says "Waiting for Recall Year.", the page says "Pick a value for Recall Year to see this report.", and the filter's button reads "Required" with a warning edge. Tiles that don't read it run as before, and setting a value runs the rest. A server or proxy that does not send required leaves everything running as before.
In the builder: a tile with no # colspan beside a sized one is drawn as wide as the reader draws it (at least 240px), not one column wide, and its width handle starts from that width; opening a report writes nothing. A report held as text that did not compile, or whose host's compileSource refused it, no longer runs its tiles into "Reference to undefined object": they are drawn with "Not run: this document did not compile.", and the problem is said once above the page.
A model's path is in the declared path field; hosts own a report's staleness and its drafts
GET …/models/{path} now sets path, the field api-doc.yaml declares, which it used to leave null while sending the same value as modelPath, a field the spec never declared. A client generated from the spec, or a proxy that re-serializes the response through it, dropped modelPath and so had no path at all. Behind such a proxy the builder's Add a tile previewed every view against models//query (a 405) and could write an import of "" for a source the report did not import yet. modelPath is still sent, deprecated: read path. The builder also no longer depends on either: it uses the path it asked for, so it works behind such a proxy even against an older server.
Report no longer overrides a host's staleness for the report key. Without manifestStaleTime, it used to pass staleTime: 0 explicitly, which beat a host's queryClient.setQueryDefaults(["report"], …), so a host that reads the manifest itself and then mounts Report over the same entry read it twice. Now the query client's default decides. With no default that is still 0, so a reader that sets nothing sees no difference. A host whose QueryClient sets a global staleTime should know that Report now inherits it; pass manifestStaleTime to fix a value for the reader.
A host that holds a report as text can give it filters of its own. ReportEditor's textSource takes compileSource(text), the text to compile in place of the document's own. The server's compile refuses a given: in held text, so a host passes a function that removes the document's given: declarations and ##! lines and writes each $NAME as a literal (malloyLiteral and localGivenSpec are now exported for this). With it, "Add filter" is on for held text: the editor saves the document's own text, with its given: lines, so it stays a report a package can serve, and the builder runs each tile with the filter's value written in as a literal, never sent as a given. textSource.takenGivenNames lists names a filter the builder adds must not take (the model's givens, including one only a gate reads), and DashboardView takes literalReads, the filters each tile reads through values the host writes into its query, which count as read for the "Doesn't respond to" chip and are never sent. Without compileSource, nothing changes. In the builder, a filter the report declares that the server has not compiled yet now applies its declared default until a value is set, as the server applies it once the report is saved; it used to filter nothing.
BuilderBar no longer widens a host's header. Its save status's screen-reader announcer was meant to be a hidden 1px box, but was sized in sx units that read as 100% wide, so in a host whose header does not clip it, the page scrolled sideways. It is 1px now.
clearReportDraft and reportDraftKey are exported from @malloy-publisher/sdk, for a host that deletes, renames or replaces a report itself and wants the builder's kept edits gone with it. clearReportDraft({ environmentName, packageName, path }) drops every draft of that report in this browser, on any server and version, and server or versionId narrow it. reportDraftKey(…) is the key the builder keeps a draft under, and clearReportDraft(key) drops that one.
A caller's extend of a gated source, or of a model named query, now follows the gate it inherits
A caller who sends source: mine is orders extend {} and run: mine got a 403 from /compile that /query did not give, even when the #(authorize) lock on orders admitted them (#1400). /compile now decides it as /query does.
/query had a related gap that /compile shared: a caller-declared derivation through a model named query or query source (source: mine is q_model extend {}, source: mine is gated -> { … }, a caller join to one) was refused for every caller, including those the gate admits, and a row-gated one never returned its filtered rows. These now admit a permitted caller, with the same rows run: q_model -> … returns, and still refuse the rest. An extend of a named query over an ungated source, previously refused, now admits any caller. That is a refusal change: a request that used to answer 403 now answers 200.
A derivation whose chain cannot be proven stays refused: a named query reached through an import, a chain past 64 names, a composite member with a row filter. See docs/authorize.md.
sslmode=no-verify in a Postgres connection string no longer breaks DuckDB attaches
A raw Postgres connectionString with sslmode=no-verify (the spelling node-pg takes for "encrypt, don't verify") failed to attach through DuckDB with libpq's invalid sslmode value. The string is now handed to libpq with sslmode=require in its place, for an attached database, a federated source and a DuckLake catalog alike. Queries through the Postgres driver still read the string as written, so no-verify keeps its meaning there.
The other node-pg-only URI spellings libpq rejects are normalized the same way: uselibpqcompat is dropped, ssl=no-verify/ssl=1/ssl=0 become their libpq equivalents, and ssl yields to an sslmode in the same string. A deployment PGSERVICE is treated like service=: the service file's sslmode is left alone.
Deployment PGSSLMODE=no-verify now means require on an attach (it previously mapped to disable, i.e. plaintext), and is applied to a raw connection string that sets no sslmode of its own. A deployment that relied on that to attach a server without TLS now fails the attach; set PGSSLMODE=disable or the connection's own sslmode for such a server.
A host can theme the report reader and builder to its own design system
ServerProvider (and ThemeProvider) take ui: the chrome's design tokens. Radii (control, tile, menu, dialog), control and chip heights, the type scale (page title, tile title, prose, control, caption, label, code), the mono face and the menu shadow, any subset, the rest staying the SDK's. Every report and builder surface reads them from the theme (theme.ui) instead of the fixed values it drew with before, so an embedding app such as Credible matches its own system by passing its values. They are the host's, not the server's, so they are a prop rather than part of the Theme object. Defaults are unchanged: a host that passes nothing sees no difference.
Stable class names for what tokens do not reach, exported as reportClasses: the report, its header and title, the filter row, each filter and its popover, each tile and its heading, a clickable cell, the builder, its bar and save status, a tile in the builder and its width handle, the builder's menus, and each builder window. These names are API and kept across releases; MUI's own class names are not.
CONTROL_RADIUS, TILE_RADIUS, DIALOG_RADIUS, CONTROL_HEIGHT_PX, CHIP_HEIGHT_PX and TYPE remain, deprecated, as the defaults. AppDialog takes className and draws its corners in ui.radius.dialog. See docs/theming.md.
The builder's windows: Add a tile, Add a filter and Clickable cells, redrawn
Add a tile is every view in the package in one searchable list, grouped by source, each by its title and Malloy name with its chart's icon; the arrow keys move and Enter adds. The picked view runs beside it as the tile it would add, with the page's filters, over its Title and a row of only the charts it can take. The Query / Text toggle is gone: Add text instead is in the footer. Add a filter shows each field's type, offers the control as five pictured kinds (one the field cannot take is greyed with the reason, rather than chosen and then refused), sums up which tiles it applies to until Choose tiles opens the list, and previews the filter's button. Clickable cells (it was titled Drill) is a switch per dimension, with "Opens … sets …" beside each one switched on. The Add menu and the tile menu are drawn as the builder's other cards.
New report is drawn the same way: every view across the package's models in the same searchable list as Add a tile (grouped by source, each group named ""<title> (<view>)" in the group of that name, and a layout from the "Layout" radio group.
The line under every report saying where Save writes is shown only when it says something out of the ordinary (Save off, a pinned version, a copy that could not be read); the builder carries data-saves-to instead, for a host or a test to read.
For an SDK host: ReportBuilder takes renderViewPreview for the picker's preview (ReportEditor passes one); AppDialog takes initialFocus, a ref focused once the window has opened, since a child's autoFocus loses to the focus trap. A test that drove the picker's Source dropdown and "View x" buttons picks an option named "<title> (<view>)" instead; one that chose a filter kind from the "Filter kind" dropdown clicks the radio of that name.
Results stay on screen while they re-run, and switching to edit no longer blanks the report
A result that re-runs keeps its previous answer up until the new one lands. Changing a filter used to drop every tile on the page to "Running…" and draw it again from nothing; now each keeps its rows, eased back after a moment, with a thin progress bar along its top, and the new rows replace them in place. This is in useQueryResult (react-query's keepPreviousData), so it holds for every surface that runs a query through ResultPanel: report tiles, QueryResult, model cells. A host test that waited for "Running…" after a change of givens waits for the new rows instead; the panel is aria-busy meanwhile.
In the Console, Edit and Done swap the page once, drawn. The page being switched to renders out of sight until its tiles in view show their results, then replaces the one on screen in the same frame, so the report is never blank in between (it was, two or three times, on the way into the builder). The builder's bar comes into the header with it. While both are mounted the page carries data-switching.
In the builder: retitling, resizing or moving a tile no longer re-runs its query, and a press anywhere off the selected tile or description clears the selection.
The report builder's controls move to the header; a host can place them itself
The builder's page-level controls are one bar in the header. In the Console, where Edit sits on a report, the builder draws its save status, undo and redo, the Linear / Grid switch, an Add menu (a tile, text or a filter) and Done. There is no Save button: edits are kept in this browser as they are made (Draft saved), and Done writes them and shows the report. Leaving any other way loses nothing and no longer asks; opening the report for editing again puts the edits back, with a note and Discard edits. Cmd/Ctrl+S still writes without leaving. A test or automation that clicked Save presses Cmd/Ctrl+S, or clicks Done. The page under it is the report as it reads: no toolbar on the title's line, no pencils after every title. Done returns to the report and asks first when edits are unsaved, as View did; View is gone from the builder.
On the page itself:
- Text is edited where it stands. A title, subtitle, description or text tile takes a faint wash under the pointer, in place of a pencil. A text tile opens in place in the report's own type, with a ring round it, rather than as an outlined box with Done and Cancel under it: leaving it or Cmd/Ctrl+Enter keeps the edit, and Escape now drops it (it used to keep it), as a title's Escape already did.
- The tile menu is a menu: Chart, a submenu, Clickable cells…, and Duplicate, which writes the copy as a view of its own (
view: by_category_tile is by_category) right after the original, so its title, chart and width can differ. A tile whose query is written in the file itself, or a pipeline, cannot be copied, and its Duplicate says so. A tile's width stays a drag of its right edge, or the arrow keys on that edge. A choice that cannot apply stays listed, greyed, with its reason under it. The source and view are a caption at its foot. - The width handle is a grip tab at the middle of the selected tile's right edge. While it is dragged, a label says the width it will snap to (
8 of 12 · Two-thirds), and the columns are drawn as faint bands behind the tiles. - A tile moves from its heading and padding, never from a press inside its table or chart, which scrolls, selects and shows tooltips as it does in the reader.
What an SDK host sees:
ReportEditorandReportBuildertakeonControlsChange. Passed, the editor draws no toolbar and hands its page-level controls over asBuilderControlswhenever they change (the save status andsave,undo/redo,layout, theaddactions, andexitwhenonExitis set), with stable actions, andundefinedwhen it goes away.BuilderBardraws them as the Console's bar. Both are exported from@malloy-publisher/sdk/builder. A host that passes nothing keeps the toolbar on the title's line, unchanged.- Drafts and Done, for a host.
ReportEditortakeskeepDraft(andReportBuilderadraftKey): unsaved edits are kept inlocalStorageper report and restored when it is next opened for editing, as one undoable edit. With a draft kept,onDirtyChangereports only text still being typed into an open field.BuilderControlsgainsdone(write, then leave once the write lands; present withonExit),draftKeptAtanddiscard, andBuilderBardraws Done in place of Save whendoneis there.draftStorageAvailable()says whether the browser keeps drafts at all, for a host deciding whether it still needs its own leave guard. - Escape in an open text tile drops the draft, and its Done and Cancel buttons are gone. A test or automation that clicked Done presses Cmd/Ctrl+Enter, or moves focus out of the field; one that clicked Cancel presses Escape.
- The tile menu's Viz type field is gone: its choices are the Chart submenu's
menuitemradioitems, and Drill is Clickable cells….
Server: package loads share the event loop with other requests, queries reuse each model's query model, and a load-phase metric
Package loads and queries do less on the main thread. Nothing an API or MCP caller sees changes: the same plans, findings, warnings and answers.
- Each loaded model builds its Malloy query model once and reuses it for every query, and for the render-tag checks the main thread prepares itself (those the load worker did not reach), where each used to build and drop its own. So does each variant of a model a query compiles against: a row-level
#(authorize)graft and the storage serve shape. Memory rises by that one structure per loaded model: about 5 MB of heap for the medium generated test package (8 model files) and 14 MB for the large one (24). A cached variant holds one too, and those caches are bounded. - Render-tag validation, model hydration and the main-thread build-plan derivation yield to the event loop every 10 ms, so a request that arrives during a load waits for at most one step rather than for the whole load.
- A model's report facts are derived once rather than on every query.
New metric: publisher_package_load_phase_duration_ms{phase}, one sample per stage of a successful load: worker, hydrate, render_validation, discovery, lint, build_plan and total. Since 0.9.9 the load worker derives the build plan, so build_plan is the time to apply it, or to derive it on the main thread when the worker left it at its soft deadline. malloy_package_load_duration is unchanged and still stops before report discovery and the build plan; total is the whole load. Each stage is wall-clock time, so a main-thread stage includes the requests served while it yields. Package loads record it; reloads do not. See docs/configuration.md.
Reports written as run: cells are no longer reports (BREAKING)
A .malloy file whose artifact tag reads as linear and lists no tiles is no longer a report. That was the older linear form: run: statements with (markdown) notes between them. Publisher stops listing it, its route opens nothing, GET …/reports/{path} does not serve it, and the builder refuses to open it rather than converting it. The package names each such file with a new error finding, report-cells-removed, whose message gives the fix, and the compile check answers the same finding. The file still compiles as a model, so nothing that imports it breaks.
Who is affected: anyone with a linear report written as cells. To tell, reload the package and look for report-cells-removed in its warnings, or on the package page. To fix one, rewrite it as tiles: each prose note becomes a named ##|(markdown) name text tile, each run: becomes a view: on an extension of its source, and the tag lists them in tiles=[…] in reading order. docs/reports.md has the steps, and the bundled examples/storefront/category-review.malloy was rewritten this way. The report-layout-implicit finding is gone: a tag with no layout= and no tiles is now this case. A tag with a misspelled layout= (layout=Grid) and no tiles is named report-layout-unknown instead, and a tiles that is not a list (tiles=foo) makes the tag unreadable (notebook-artifact-unparsed). The replace write, PUT …/models/{path} with expectedHash, still takes such a file, so it can be rewritten as tiles in place.
What an SDK host sees:
Reportno longer renders cells. It draws a report's tiles throughDashboardView, and a manifest that carries cells draws none of them. Two behaviours that were particular to cells go with them: a report's startinggivens { … }apply without being written into the URL, as a grid's already did; and controls are shown only for the givens the tiles read, so a given only an#(authorize)gate reads is the host's to supply unless a tile reads it too.- The builder no longer converts.
readDashboardDocumentrefuses a cell-format file with a reason that names the fix.ReadFailure.legacyNotebook,ReportBuilder'sconversionprop, the conversion dialog, and theconvertedflag on thereport.savedevent are removed. readReportTagOfText(from/builder) can answer{ status: "cells-removed" }, withunknownLayoutset when the tag names a layout Publisher does not know. Text it used to read as{ status: "report", layout: "linear" }(a linear tag with notiles) is no longer a report, so a host that switches overReportTag.status, or keeps its own copy of the rule, needs the new case.DashboardViewtakesonLinkNavigate, for links inside text tiles and the description, asReportpasses it. Without it, relative links stay as written..malloynbsupport is unchanged and fenced off. Its renderer now lives in one folder of the SDK's legacy code, marked deprecated, and a lint rule stops anything else from importing it.Notebookstill renders a.malloynb.
A multi-pick filter keeps the value you just picked when you click away
In a filter that takes several values (control=multiselect on a filter<string> given), picking a value and then clicking anywhere else on the page removed the value again: the filter read "All" and the query ran unfiltered. The list stays open after a pick so you can pick another value, and leaving it re-picked the option still highlighted, which in a multi-pick takes it back out. A picked value now stays. Text you typed without picking from the list is still kept when you click away, as before. Single-value pickers were never affected.
Reports: one look for every layout, a row of filter buttons, and package problems on the package page
Every report draws the same page. A grid report and a linear report now draw the same header, the same filter row, query results in the same cards and prose as bare text, in the reader and in the builder alike. The layout decides only how tiles are arranged: a linear report is one full-width column with tiles inserted between others, a grid has columns and widths. Before, a linear report drew no cards, wider gaps and its own title block.
What an SDK host sees:
GivensPanelwithlayout="bar"draws a row of filter buttons (Category All,Ordered since Jan 1, 2023), each opening its control in a popover, instead of a row of form fields. A test or automation that typed into a filter's field now opens its button first: the button's accessible name is<label>: <value>, the popover is named<label> filter. Thepanellayout is unchanged.chromeonReportandDashboardViewis accepted and ignored. A host that passedchrome="none"for a linear report gets cards around its query tiles, as a grid has; text tiles and the description are prose, never a card, in either layout.- Report surfaces size their controls and tiles themselves: 32px controls, 6px control corners, 8px tiles, a 14px semibold tile title, rather than the host theme's button sizes and
shape.borderRadius. Colours still come from the host palette and the instance theme. - The report title is an
h1. An untitled linear report is titled with its file name, as an untitled grid already was. - The report page carries
data-layout(gridorlinear) in the reader and the builder alike, for a host or a test that needs to tell the two apart now that both draw cards. A test that checked a linear report by the absence of cards should check this instead. ReporttakesmanifestStaleTime, how long a read of the report is reused without asking the server again (default 0, so a change made elsewhere shows on the next mount). The Console sets a few seconds on the reader it draws while the builder loads, which is how opening the builder reads the report once.
In the builder, a report's filters are configured on the filter row itself: a remove badge on each button's corner, its settings from its popover, "+ Filter" at the end of the row, and a filter no tile reads yet shown dashed and marked unused. The row of chips above the controls is gone.
A tile is removed the same way: a × on its corner while it is hovered, focused or selected, rather than Delete in its "⋯" menu, which now holds the chart and drill only (a text tile has no menu). The last tile of a saved report keeps its × but says why it cannot go.
Switching from reading to editing no longer blanks the page. The Console keeps the report on screen while the builder's code and data load, and starts fetching the builder's code when the pointer reaches Edit. A tile the builder has not changed runs exactly as the reader runs it, so it is drawn from the reader's cached result instead of re-running. The builder also reads the report's manifest from the reader's cache entry. ReportEditor takes loading, what to draw while the report opens (default a spinner), for a host that wants the same.
The package page says what failed to load. A file the server could not serve as intended (for example one whose artifact tag still sets kind=, which 0.9.8 stopped reading, so it listed under Models rather than Reports) is named with the server's own message and fix. Lower-severity notes are folded behind a count. The Reports heading's create button is now + Report, accessible name "Add report", like every other section's add button.
Console. Buttons are 32px with 6px corners rather than pills, cards and menus 8px, dialogs 12px; breadcrumbs are text with slashes. A report, in reading and in editing, sits in the model page's frame: at most 1200px wide rather than 1600px, its title 48px under the top bar like the model's and the package's, so moving between a model and a report does not shift the page.
A where: on a composite source is served again
Since 0.8.1, a query on a composite source carrying a where: could answer 403 Access denied for source "<member>" with no gate in the package: a where: on a field the members declare, one written inside a member, and, since 0.8.3, one through a join declared on the composite. These are served again, including for a composite reached through a join. Rebinding a field such a filter reads is still refused, and so are two shapes this does not cover: a member whose own join is a composite, queried through that join, and a member dimension that reads a private member field. Upgrading is enough. See #1277.
Two composite shapes still fail, in Malloy rather than in this check: members that take parameters (Malloy drops a member's arguments when it resolves the composite), and a join_many to a composite that carries its own where: (Malloy generates invalid SQL for it).
What's Changed
- chore(release): 0.9.10 by @housejester in #1394
- feat(reports): one look for every layout and format, a row of filter buttons, package problems by @kylenesbit in #1386
- fix(sdk): a multi-pick filter keeps its pick when the reader clicks away by @kylenesbit in #1397
- feat!: retire the run: cell format for .malloy reports; fence .malloynb by @kylenesbit in #1395
- perf(server): share query models, yield during loads, time load phases by @housejester in #1387
- feat(builder): edit chrome — controls in the header, one selection ring, a grip for the width by @kylenesbit in #1398
- feat(builder): drafts and Done, smooth transitions, redrawn windows, and host theming by @kylenesbit in #1399
- fix(server): read composite fields through to the resolved member in the filter-binding check by @stefanoperenzoni in #1278
- fix(server): rewrite sslmode=no-verify to require in a postgres connectionString before DuckDB attaches it by @Sha-Bang in #1393
- fix(server): a caller-declared derivation is decided by the gate it inherits, on /query and /compile by @Sha-Bang in #1406
- fix: host integration — model path, report staleness, drafts, model-read filters, skills, and compileSource for held text by @kylenesbit in #1401
New Contributors
- @stefanoperenzoni made their first contribution in #1278
Full Changelog: v0.9.10...v0.9.11
