v0.9.8

Release v0.9.8

📦 NPM packages

• @malloy-publisher/sdk@0.9.8
• @malloy-publisher/app@0.9.8
• @malloy-publisher/server@0.9.8

🐳 Docker image

docker pull ms2data/malloy-publisher:0.9.8
docker pull ms2data/malloy-publisher:latest

ℹ️ Details

• Release branch: release/v0.9.8
• Triggered by: @kylenesbit

Reports: one document type, anywhere in a package, with a layout

Dashboards and notebooks are now one document type, the report, and a report can live anywhere in a package. A report is any .malloy file outside public/ that carries a model-level ## artifact tag, other than the root index.malloy; a .malloynb is a linear report wherever it is. dashboards/ and notebooks/ are now ordinary directory names: nothing about a file depends on which directory holds it, and a report two directories deep is served like one at the root. Every file served from those two directories before is classified the same way by its tags alone, with one exception, listed under "What may change" below.

A report's layout is grid or linear, decided by its tag: layout=grid / layout=linear; else grid when the tag lists tiles or the file carries a model-level dashboard { … } grid tag; else linear. One reader decides it, over the file's model-level notes, whether the file compiled, failed to, or is text handed to the compile check or the save, so every one of them answers the same.

A report is identified by its path. Two reports may share a basename in different directories and both are served; the old shadowing (the second file was not served) and the write path's name-collision 409 are gone.

New routes. GET …/reports lists every report with path, name (the basename, a label), layout, title, description, editable and any load error. GET …/reports/{path} returns one report by its full package path including .malloy (or .malloynb), exactly as the listing gives it: the manifest fields for a report written as tiles, or notebook (its cells) for a run:-cell report or a .malloynb. There is no report cell route: a .malloy report's run: cells run through POST …/models/{path}/query, each cell's text as the query, the governed endpoint grid tiles already use. DashboardManifest, RawNotebook.dashboard and a compile's CompiledDocument gain layout; package warnings gain an optional code. The report schemas reference nothing deprecated: ReportTile and ReportCells are the tile and cell shapes, with DashboardTile and RawNotebook as deprecated aliases. A write whose rollback itself fails answers 500. Severities are unchanged: every new finding is warn or error, so a client that accepts only those keeps working.

Writing. PUT …/models/{path} (the builder's save) now creates a report at any .malloy path outside public/ other than the root index.malloy (refused whatever its case), creating missing directories inside the package root only, and replaces only a file the package reads as a report. It still cannot overwrite a model, an untagged file, a .malloynb or index.malloy. Text whose only ## artifact is inside a comment or a string is refused with 400 up front, and a link out of the package is refused before anything is read through it. See docs/security-posture.md.

SDK. createDocument (and the New report dialog) writes a new report at the package root by default, so CreatedDocument.path is sales.malloy where it was dashboards/sales.malloy or notebooks/sales.malloy, and the template always writes layout=grid or layout=linear in the tag. A host whose DocumentStorage files documents somewhere other than the path asked for (every grid report under dashboards/, say) implements the new optional resolveDocumentPath(locator), returning the package-relative path saveDocument will really write: the new file's imports are rendered for that path and CreatedDocument.path is that path. A store without it is assumed to write where asked; on such a store that relocates files, a new report's imports would not resolve. Removed exports: documentPathFor(kind, slug) and documentPathForTitle(kind, title, suffix?), which named dashboards/<slug>.malloy / notebooks/<slug>.malloy, are gone from @malloy-publisher/sdk/builder. Use reportPathFor(slug, directory?) and reportPathForTitle(title, suffix?, directory?), which name a report's file at the package root (or in directory); a host that files reports by kind builds its own path. createDocument checks every path it computes, a directory included, so directory: "../x" or "public" is refused before the package or the store is asked anything. ReportEditor and ReportBuilder take allowLayoutChange (default true); false hides the Linear / Grid switch, for a host that files reports by layout. The Console's builder route is now the report's route with ?edit (/<env>/<pkg>/sales/q3?edit), so a report whose path ends in edit opens as itself; reportEditRoute(env, pkg, path) builds it, and the package page's onOpenDocument default and drafts use it. A …/sales/q3/edit link still opens the builder by redirecting to ?edit, unless a report is at sales/q3/edit.malloy. A report under data-apps/ opens as a report rather than as a data app. The builder reads a file's layout as the server does, from its model-level notes alone, never its folder: layout=, else grid when the tag lists tiles or the file carries a model-level dashboard { … } tag, else linear; a tag that sets kind= opens nothing, and the builder names the layout= to write. readReportTagOfText(text) (@malloy-publisher/sdk/builder) gives that answer for any text.

SDK changes a host must make (all to APIs the new format introduced or to the storage contract):

• DocumentStorage: a report's path is its identity. DocumentLocator.path (<env>/<pkg>/<package path>) is the key: one path is one document. DocumentLocator.type (DocumentType) is now the report's layout, "grid" or "linear", metadata a save records and a listing reports, where it was "dashboard" or "notebook" and part of the key. Migration for a host store: key documents by path alone, record type on save and move, report it from listDocuments, and read "grid" for an old "dashboard" and "linear" for an old "notebook". BrowserDocumentStorage moves drafts saved under its old type-keyed scheme to their paths the first time it is used, so none is lost. useDraftType is now useDraftLayout, answering { layout, settled }.
• The builder's document carries layout (DashboardDocument.layout, "grid" or "linear") in place of kind. Migration: read document.layout === "linear" where you read document.kind === "notebook". SpliceOptions loses changeKind, modelPath and explicitKind, and ReportBuilder loses explicitKind: none changed anything any more.
• ReportEditor takes layout, the layout a report opens with (from the listing, or a draft's recorded layout); kind still works, deprecated. Once open, the document's own layout decides how controls are drawn, which layout a save records, and which events the builder reports, so they follow a layout switch.
• createDocument takes layout only: its deprecated kind option, CreatedDocument.kind and OpenDocumentRequest.kind are gone, and newDocumentProblem(input) takes no kind. LocatedDocument has layout and no kind. Migration: read layout.
• Drill resolves paths only. resolveDrillTarget answers { kind: "path", report } or { kind: "unresolved" }; the by-name arm and candidates are gone, and the Console never builds a /dashboards/<name> link for a drill. locateReportByName, locateDocument and documentRoute remain, deprecated, for old links only.
• Report renders .malloy reports; a .malloynb is rendered by the deprecated Notebook, the only component that calls the deprecated notebook routes. The Console routes a .malloynb there.
• reportRoute encodes the environment and package names as well as the path, so no caller encodes them again.

Server telemetry, renamed outright (no dual emission). Dashboards to update:

• publisher_dashboard_writes_total → publisher_report_writes_total, and publisher_dashboard_write_duration_ms → publisher_report_write_duration_ms. The kind label is now layout (grid/linear); outcome keeps its values and gains rollback_failed (a write whose restore failed too, so the package may serve either text); the counter gains reason, why a write did not land (frozen_config, invalid_path, missing_source, untagged, not_a_report, conflict, compile_failed, not_found, reload_failed, bad_request, other; none when it did).
• publisher_notebook_discovery_total is removed. Discovery is now counted for reports of both layouts: publisher_reports_discovered_total{layout}, publisher_report_files_total{status} (served, tileless, invalid, dropped) and publisher_report_lint_findings_total{code,severity}, once per load or reload, with publisher_report_discovery_duration_ms{during} (load, reload, compile) timing every pass. A notebook whose cells the reader refused, its old outcome="refused", is publisher_report_lint_findings_total{code="notebook-cells-unreadable"}; a .malloynb load is publisher_deprecated_usage_total{feature="malloynb"}.
• New: publisher_report_read_duration_ms{route,status_class} for GET …/reports (list) and GET …/reports/{path} (get), and publisher_deprecated_usage_total{feature}, one per use of a deprecated feature: the legacy routes (dashboards_list, dashboard_get, notebooks_list, notebook_get, notebook_cell), the Console's /dashboards/<name> alias (console_alias), and per load or reload a .malloynb file (malloynb), a package setting explores or queryableSources (explores, queryable_sources), a source declaring #(filter) (filter_annotation), and a query sending filter values or a bypass (filter_params, bypass_filters). It is the evidence for when each can go.
• publisher_notebook_cell_executions_total and publisher_notebook_cell_execution_duration_ms are unchanged; they measure the deprecated cell route only.

Logs: report lint findings are no longer logged at warn on every load, reload and metadata change. Each load or reload logs one info line, Report discovery, with the package, during, the counts by layout and status, and the findings by code; each finding is a debug line, Report lint finding, with code, severity and model. Report discovery and write log lines name the file model (was modelPath) and the failure error (was detail). A rolled-back write is logged once (Report write rolled back, warn), and a failed rollback once at error (Report write rollback failed) with both error and restoreError; a refused write is Report write refused at warn. A 4xx on the report and legacy report routes is logged at warn, not error.

SDK telemetry, renamed outright (no dual emission). Every report surface reports one event family, ReportEvent, through the onEvent props it already had, and every event carries the report's package path and its layout (grid/linear; absent only on a report.load_failed that happened before the report said). The host adds the environment and package when it logs, as before.

• Removed SDK types: DashboardEvent, DashboardEventHandler, NotebookEvent, NotebookEventHandler, BuilderEvent, DocumentCreatedEvent, DashboardCreatedEvent and NotebookCreatedEvent. Use ReportEvent and ReportEventHandler (both entries), with ReportEventContext, ReportFailure, DrillOutcome and SavesTo.
• Renamed events: dashboard.opened / notebook.opened → report.opened (surface: "edit"; cells is now tiles); dashboard.open_refused / notebook.open_refused → report.load_failed with a category (not_found, forbidden, invalid, unsupported, server, network, unknown) and no reason, since the message can carry a path or a value; dashboard.saved / notebook.saved → report.saved (cells → tiles; converted is present only when true); *.save_refused, *.save_undone, *.save_undo_refused → report.save_refused, report.save_undone, report.save_undo_refused; dashboard.created / notebook.created → report.created (now with durationMs); dashboard.rows_shown → report.rows_shown; dashboard.explored → report.explored. Migration: match on the report.* name and read layout where you read the dashboard./notebook. prefix.
• New events: report.opened and report.load_failed from Report itself (surface: "view"), once per report shown, for grid, linear and run:-cell reports alike, where before only a grid report's rows dialog reported anything; report.query_failed (tile index, category) for a tile or cell whose query failed; report.create_refused (reason) for a create that did not land; report.layout_changed (from, to) when an author switches Linear/Grid; report.drill (to: "self" | "report", outcome: "resolved" | "unresolved" | "malformed").
• Prop changes: Report's onNavigate may return a DrillOutcome, which Report reports as report.drill; returning nothing reports nothing. Package takes onEvent and passes it to its New report dialog. DashboardView no longer takes onEvent: it reports through the Report (or deprecated Dashboard) that renders it. Inside a Report, a failed cell and a refused to=self drill go to onEvent and are no longer logged to the browser console; without an onEvent they still are.
• Console: every event is one log line under [publisher.report] (was [publisher.dashboard] and [publisher.notebook]), at warn for a failure (load_failed, query_failed, any *_refused, a failed rows_shown, a drill that did not resolve) and info otherwise.
• A host with an event sink must update it before taking this SDK. Credible's draft editor matches dashboard.saved, notebook.saved, dashboard.save_undone, notebook.save_undone, dashboard.created and notebook.created, and imports BuilderEvent; it needs report.saved, report.save_undone and report.created, and ReportEvent.

Deprecated, still working until a later release removes them:

• GET …/dashboards and …/dashboards/{name} (the grid reports) and GET …/notebooks, …/notebooks/{path} and …/cells/{i} (the linear reports). …/dashboards/{name} resolves a basename only when exactly one grid report has it, and answers 409 naming every match when several do. Move to …/reports.

  These deprecated routes still answer, but their contracts are wider than before, and a client of them should check:

  • GET …/dashboards/{name} can now answer 409 (several grid reports share the basename). It never did before. The package warns about each such basename at load (report-name-ambiguous, warn, naming the paths); fix it by renaming one of the files.
  • GET …/dashboards and GET …/notebooks now list reports from anywhere in the package, not only files directly under dashboards/ or notebooks/: a path may be sales/q3.malloy or a/b/c.malloy, and a grid report in notebooks/ is listed by …/dashboards, a linear one in dashboards/ by …/notebooks.
  • Dashboard.name is still the basename, but it is no longer unique in the listing; key on path.
  • The SDK's DrillNavigation.path (what a # drill click hands a host; dashboard before, see below) is the tag's to=, which is now usually a path containing / (sales/overview) rather than a single-segment slug. Resolve it with resolveDrillTarget against GET …/reports rather than putting it into a URL as one segment.

• SDK names that said "dashboard" for a report, renamed with the old names kept as deprecated aliases, removed with the other deprecated APIs. Migration: import the new name; nothing else changes.

  • DashboardEditor → ReportEditor and DashboardBuilder → ReportBuilder (@malloy-publisher/sdk/builder), with DashboardEditorProps → ReportEditorProps and DashboardBuilderProps → ReportBuilderProps. The old names are the same components and types.
  • DrillNavigation.dashboard → DrillNavigation.path. Both are set, to the same value, on every drill.

Removed, not deprecated. These existed only for the new format, which has no users yet, so they go now rather than through a deprecation window:

• kind= in an artifact tag is no longer read. A tag that sets it makes no report, and the package reports report-kind-removed (error). Migration: write layout=grid for kind=dashboard, layout=linear for kind=notebook, and drop kind=.
• The query-level # artifact on a query: is no longer read; only a model-level ## artifact makes a report, and a file whose only tag is on a query is an ordinary model. query-artifact-deprecated, query-artifact-ignored and the manifests' query field go with it. Migration: ## artifact { layout=grid <the same properties> tiles=["<query>"] } at model level, with the query's # dashboard { … } left on the query.
• A # drill { to=… } target is always a path from the package root; a bare name is a root file. The basename fallback and drill-target-by-name are gone. Migration: write the report's path, to="sales/overview" for sales/overview.malloy.

What may change in an existing package. A model outside dashboards/ and notebooks/ that carries an artifact tag is now a report: it leaves the model listing on a curated package and contributes nothing to the surface. The package says so with report-on-surface whenever index.malloy imports it or explores lists it. A tagged root index.malloy stays the surface, and its tag is ignored with an index-is-not-a-report error. A kindless, tileless ## artifact in dashboards/ (an unserved empty dashboard before) is now a linear report, reported as report-layout-implicit. A notebooks/x.malloy whose ## artifact lists tiles and names no kind= or layout= read as a notebook before, because the folder supplied the kind; it is now a grid report, so it moves from …/notebooks to …/dashboards and renders as a grid. Nothing warns about it: add layout=linear to keep such a file linear. A file whose artifact tag sets kind=, or whose only tag is a query-level # artifact, is no longer a report (see Removed above). report-on-surface follows index.malloy's imports transitively, as the surface does. notebook-other-folder and notebook-kind-missing are retired, and a package's own public/dashboards/ and public/notebooks/ files are reachable at /<env>/<pkg>/dashboards/<file> again.

What's Changed

• chore(release): 0.9.7 by @housejester in #1372
• feat(server): give the deprecated-format package warnings a stable code by @kylenesbit in #1371
• feat: reports — dashboards and notebooks as one document type, anywhere in a package by @kylenesbit in #1373
• test: move tests, fixtures and examples off deprecated features by @kylenesbit in #1376
• refactor: reports quality pass — simpler code, isolated deprecations, test pyramid, telemetry by @kylenesbit in #1377

Full Changelog: v0.9.7...v0.9.8