viewer/compare: pin baseline left + show only populated dim columns by changliu2 · Pull Request #180 · responsibleai/ASSERT

changliu2 · 2026-06-01T04:22:06Z

viewer/compare: pin baseline left + show only populated dim columns

TL;DR

Three independent UX/correctness fixes on the compare and suite-landing views, in three commits:

viewer/suite — render columns from the eval's actual dimensionDefs instead of three hardcoded headers (Policy violation, Overrefusal, Harm actionability). For evals with custom dimension names (e.g. safety_violation, unjustified_refusal), the previous UI silently relabeled the customer's metric and showed a dead — column for the unsupported harm_actionability. Headers now come from Object.keys(data.dimensionDefs), per-row values from metrics.dimensions[name].rate, and any column where every visible row is null is hidden. No cap — the table scrolls horizontally like the by-behavior heatmap.
viewer/compare — pin the baseline run to the leftmost column across summary cards, the by-behavior heatmap header, and per-prompt sample columns. Tagging a card Baseline while leaving it on the right inverted the diff-tool convention (baseline left, change right) every reader expects; -18% deltas read as "left is better than right" on first glance when they actually mean "left is better than baseline on the right." Colors are now keyed by run_id via a runColor map so a run keeps its color regardless of column position. ?runs= URL order is preserved as the canonical user-selection order.
viewer/compare — sort the per-prompt sample panels with a three-key comparator: (baseline-flagged DESC, has-disagreement DESC, |Δ| DESC), tiebreak on prompt. Lead with the regression-fix story (baseline-fail / variant-clear at top) without hiding regressions (baseline-clear / variant-fail next). The previous sort was disagreement-first then scoreSortValue on whichever sample happened to be first non-null, which made screenshot order subtly non-deterministic when the baseline was missing a prompt.

Decisions worth flagging

No column cap on the suite-landing dim table. The spec floated capping at 5; dropped because the by-behavior heatmap already overflows horizontally and a cap forces an overflow UI we don't need. If a customer has >7 dims we'd rather they scroll than introduce a kebab-menu column picker now.
Sort key 1 is baseline-flagged = getRecordFlag(sample, metric) === true. I treated judge-failure rows as not-flagged so they fall to the bottom (rather than being treated as agreement-on-clear). Matches the per-spec acceptance criterion.
Aggregate helpers aggregateRunViolationRate / aggregateRunOverrefusalRate are left in place even though the table no longer calls them — they're tiny and may have downstream callers. Will sweep in a follow-up if confirmed dead.

How to verify

Existing bank-manager ACS demo artifacts in artifacts/results/bank-manager-agent-shield/ exercise all three changes:

Open the suite landing page → the table should show columns for safety_violation and unjustified_refusal (the eval's actual dims) rather than the old hardcoded three. No empty Harm actionability column.
Click Compare → pick a baseline → it should render leftmost across the cards, the heatmap header, and inside any expanded behavior row's sample panel.
Expand a behavior with mixed outcomes → baseline-flagged-variant-clear prompts (the "wins") should be at the top; regressions next; agreement at the bottom.

CI / typecheck

npm run check in viewer/ reports the same 3 pre-existing errors (PrimerPagination.svelte rel prop, routes/+page.svelte sortBy typing) on main as on this branch. Zero new errors introduced.

Out of scope

Issue 3 (Prompt | Scenario segmented control + paired-transcript view) is M effort and needs the divergent-transcript view to ship honestly; deferred to a separate PR.
Run-detail page (/suite/[id]/[run_id]) has its own column conventions — a follow-up audit can align landing and detail.

Fixes correctness bug where the evaluation-results table hardcoded three columns (Policy violation, Overrefusal, Harm actionability) regardless of what the eval actually populated. Custom dim columns were silently relabeled and a dead column appeared for unpopulated dims. Now derives column headers from data.dimensionDefs, reads each row's value from metrics.dimensions[name].rate (via aggregateRunDimensionRate for parent rows), and hides any column where every visible row is null. No cap on column count — the table scrolls horizontally like the by-behavior heatmap already does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…tmap, and per-prompt panels The baseline dropdown previously tagged a run as `Baseline` but did not move it. Result: in a typical 2-run compare the baseline often rendered on the right while the variant under test rendered on the left, inverting the diff-tool convention every reader expects (baseline left, change right). Delta signs (-18%) then read as `left is better than right'' on first glance when they actually mean `left is better than baseline on the right.'' Now derives `orderedRuns = [runs[baselineIdx], ...others]` and feeds it to all three render loops (summary cards, heatmap header, per-sample columns). Colors are now keyed by run_id via a `runColor` map so a run keeps its color regardless of column position. `?runs=` URL param order is preserved (it represents user selection, not display order); switching baseline does not rewrite the URL. `baselineIdx` semantics still index `data.runs`, not `orderedRuns`. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ement to lead with the regression-fix story When a reviewer expands a behavior row, the per-prompt sample panels previously sorted by (disagreement-first, then by `scoreSortValue` on the first non-null sample). For the dominant demo case (baseline failed N prompts; variant fixed them), this buried the win story below high-magnitude regressions and judge-failure rows. Re-sorts with a three-key comparator: 1. baseline-flagged DESC (the wins demo case — only applied when `baselineRunId` is provided) 2. has-disagreement DESC 3. |delta| on active metric DESC tiebreak: prompt for stable, reproducible order across reloads Puts baseline-fail-variant-clear panels at the top, baseline-clear- variant-fail panels next, and pure agreement at the bottom. Honest about regressions while leading with the narrative. `baselineRunId` is an optional new param; when omitted (non-compare contexts) the sort falls back to disagreement/delta/prompt and behaves the same as today's disagreement-first behavior modulo the tiebreak. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…s table The populated-dim fix in 7dea023 derived column headers from data.dimensionDefs and filtered out empty columns — but dimensionDefs only contains BUILT_IN_DIMENSIONS + the global examples/eval-definitions/judge_dimensions.yaml file. Custom dimensions declared inline in a suite's eval.yaml judge.dimensions block (e.g. safety_violation, unjustified_refusal in the bank-manager-agent-shield demo) never reach loadDimensions(), so they were silently dropped from the suite-landing table even though they appear in every run's metrics.dimensions. Now derives the dim-name set as the union of (a) dimensionDefs keys and (b) Object.keys(run.{prompt,audit}.metrics.dimensions) across all runs. Mirrors the pattern loadComparePageData already uses (allMetrics = union of Object.keys(summary.dimensions)). visibleDimNames keeps the existing filter that drops dims with no data, so the harm_actionability column still hides for evals that don't score it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

changliu2 · 2026-06-01T05:39:07Z

Added commit a2658eb — extends the populated-dim column logic to also surface custom per-suite dim names in the suite-landing Evaluation results table.

The fix in 7dea023 derived headers from data.dimensionDefs and filtered out empty columns, but dimensionDefs only contains BUILT_IN_DIMENSIONS (policy_violation / overrefusal / harm_actionability) plus the global examples/eval-definitions/judge_dimensions.yaml file. Custom dims declared inline in a suite's eval.yaml judge.dimensions block (e.g. safety_violation and unjustified_refusal in the bank-manager-agent-shield demo) never reach loadDimensions(), so they were silently dropped from the suite-landing table even though they appear in every run's metrics.dimensions.

Now derives the dim-name set as the union of (a) dimensionDefs keys and (b) Object.keys(run.{prompt,audit}.metrics.dimensions) across all runs — mirroring the pattern loadComparePageData already uses (allMetrics = union of Object.keys(summary.dimensions)). visibleDimNames keeps the existing filter that drops dims with no data, so harm_actionability still hides for evals that don't score it.

Verified locally against artifacts/results/bank-manager-agent-shield/ — both variant-a-unguarded-n100 and variant-e-guarded-acs-n100 write all four dim keys to metrics.dimensions (e.g. safety_violation: {rate: 0.22, count: 50, ...} in the variant-a prompt run), so the table now picks them up and renders Safety violation / Unjustified refusal columns alongside Policy violation / Overrefusal.

npm run check reports the same 3 pre-existing errors documented in the PR body. Zero new errors.

…d to AssertAi* (#187) Catches up TS symbol names with PR #177 (assert_eval -> assert_ai package rename) and PR #182 (ASSERT_EVAL_* -> ASSERT_AI_* env var rename). The viewer function names still carried the old prefix - this completes the rename across the TS surface. No behavior change. Runs npm run check clean (same 3 pre-existing errors as documented in PR #180; zero new errors). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The target is invariant within a run group, so showing it 3x per group (aggregate row + Prompts subrow + Scenarios subrow) wastes horizontal space. Drop the standalone Target column and surface target as a secondary monospace line under the run name link in the aggregate row. Frees a fixed column slot for dimension columns. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Each of the up-to-3 metric column headers is now a <select> that lets the user pick which populated dim that column shows. Defaults to the first 3 populated dims (preserving the existing visibleDimNames order introduced in a2658eb). The choice persists in the URL search param `?metrics=<comma-list>` so links are shareable and reloads keep the column selection. Selecting a dim already shown in another column swaps the two slots so the same dim is never duplicated across columns; options for already-used dims are also marked disabled in other dropdowns as a visual hint. The Total column is unchanged. Cells for runs that don't have a chosen dim continue to render the em-dash `—` placeholder. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

changliu2 · 2026-06-01T06:49:55Z

Stacked 2 more commits on this PR addressing follow-up evaluation-results table UX issues:

afd95bf — refactor(viewer/behavior-table): move target into run name cell. The target string was being rendered once per subrow (aggregate row + Prompts subrow + Scenarios subrow), repeating the same value 3x within every run group. Dropped the standalone Target column and surface the target as a secondary monospace line under the run name link. Frees one fixed column slot for dim columns and removes ~30% of horizontal noise on demos with long target identifiers like examples.bank_manager_agent_shield.agent:chat_guarded_acs.
d74e009 — feat(viewer/behavior-table): dimension columns are now dropdowns. Each of the up-to-3 metric column headers is now a <select> populated from visibleDimNames (the same data source the populated-dims fix in a2658eb already derives). Default = first 3 populated dims in their existing order, so the table renders identically to before on first load. The selection persists in the URL search param ?metrics=<comma-list> for shareable links. Selecting a dim already shown in another column swaps the two slots so the same dim is never duplicated; options for already-used dims are also disabled in other dropdowns as a visual hint. The Total column is unchanged. Em-dash — placeholder logic from a2658eb is preserved when a run doesn't have the chosen dim.

CI: npm --prefix viewer run check reports the same 3 errors / 6 warnings baseline as this PR's prior commits — zero new diagnostics in the edited file.

…aryLabel hint Adds two optional fields to DropdownOption: - disabled: renders the option with reduced opacity. Click handlers still fire so callers can implement swap-style behavior (discoverability over prevention). - secondaryLabel: muted, smaller-font inline hint shown next to the main label. Useful for explaining why an option is in a disabled visual state (e.g. 'in column 2'). Backward compatible: existing callers that omit both fields render identically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…kers Replaces the three native <select> column headers (added in d74e009) with PrimerDropdown instances. The native dropdown had no visual differentiation between the option currently selected in this column, options currently selected in other columns, and freely available options. Now uses the same dropdown component as the compare page Metric picker, with: - bold/highlighted = selected in this column - normal = available - faded + '(in column N)' hint = currently selected in another column, still clickable to trigger the existing swap-on-click behavior in setMetricCol URL ?metrics= state and swap-on-click semantics are preserved. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add an optional 'kind' parameter to loadComparePageData so the compare page can request scenarios (multi-turn) instead of prompts (single-turn). When a run has zero scenario samples, the page server falls back to the prompts payload metadata and marks the response with emptyKind=true so the UI can render a 'No scenarios in these runs' empty state with the toggle still visible. buildMatchedSampleRows now accepts an optional keyFn so scenario rows can be paired by test_case_id while keeping the default prompt-text key for prompts mode. The display-side 'prompt' field always holds the first user turn. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The compare page now shows the same Prompts/Scenarios segmented control as the single-run view, bound to a new ?kind=prompts|scenarios URL param. Default stays prompts so existing compare links are unchanged. Scenarios mode renders each behavior row with the shared opening (seed) prompt as the header, then each run's full multi-turn transcript in a scrollable per-run panel. Conversations are matched by test_case_id across runs because the seed turn is identical but the rest of the conversation diverges as the adaptive tester drives each variant differently. The existing baseline-pin and baseline-flagged-first ordering carry over to scenarios mode untouched — buildMatchedSampleRows just receives a different key function. When a run has no scenario samples, the page renders a 'No scenario evaluations recorded for these runs' empty state with the toggle still visible so the user can switch back to Prompts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds scenarios (multi-turn) support to the suite compare view and makes metric selection more configurable/shareable via URL params.

Changes:

Introduces kind=prompts|scenarios on the compare page, with a scenarios empty-state fallback.
Updates compare rendering to support baseline-first ordering, run-id keyed colors, scenario transcript display, and improved sample matching.
Adds URL-driven metric column selection to the suite page and extends PrimerDropdown with visually-disabled options.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
viewer/src/routes/suite/[suite_id]/compare/+page.svelte	Adds kind toggle, scenario rendering, baseline-first ordering, run-id keyed colors, and updated sample pairing.
viewer/src/routes/suite/[suite_id]/compare/+page.server.ts	Parses `kind` param and returns a scenarios empty-state payload when needed.
viewer/src/routes/suite/[suite_id]/+page.svelte	Adds URL-driven selection for metric columns in the runs table.
viewer/src/lib/server/data.ts	Extends `loadComparePageData` to load prompts vs scenarios data.
viewer/src/lib/compare-view.ts	Adds optional matching key function and baseline-aware sorting for matched sample rows.
viewer/src/lib/PrimerDropdown.svelte	Adds `disabled` + `secondaryLabel` rendering to dropdown options.

+	const url = new URL(page.url);
+	if (kind === 'scenarios') url.searchParams.set('kind', 'scenarios');
+	else url.searchParams.delete('kind');
+	goto(url.toString(), { replaceState: true, noScroll: true, keepFocus: true });


+		const url = new URL(page.url);
+		url.searchParams.set('metrics', next.filter((n): n is string => Boolean(n)).join(','));
+		goto(url.toString(), { replaceState: true, noScroll: true, keepFocus: true });


+        <li class="ActionList-item" role="option" aria-selected={selected === option.value} aria-disabled={option.disabled || undefined}>
          <button
            type="button"
-            class="ActionList-content w-full text-left {selected === option.value ? 'ActionList-content--selected' : ''} {highlightedIndex === idx ? 'ActionList-content--highlighted' : ''}"
+            class="ActionList-content w-full text-left {selected === option.value ? 'ActionList-content--selected' : ''} {highlightedIndex === idx ? 'ActionList-content--highlighted' : ''} {option.disabled ? 'ActionList-content--disabled' : ''}"
            onclick={() => handleSelect(option.value)}


+	// Three-key sort to lead with the regression-fix narrative without hiding regressions:
+	//   1. baseline-flagged DESC (baseline-fail rows first — the "wins" demo case)
+	//   2. has-disagreement DESC (mixed-outcome rows next)
+	//   3. |delta| on the active metric DESC (largest spread first)
+	//   tiebreak: prompt for stable, reproducible ordering across reloads


+	const absDelta = (row: MatchedSampleRow): number => {
+		const scores: number[] = [];
+		for (const sample of Object.values(row.samples)) {
+			if (!sample) continue;
+			const flag = getRecordFlag(sample, metric);
+			if (flag === true) scores.push(1);
+			else if (flag === false) scores.push(0);
+		}
+		if (scores.length < 2) return 0;
+		return Math.max(...scores) - Math.min(...scores);
+	};


+		const result: (string | null)[] = new Array(numCols).fill(null);
+		const used = new Set<string>();
+		if (raw) {
+			const parts = raw.split(',').map((s) => s.trim());


+		if (existingIdx >= 0) next[existingIdx] = next[colIdx];
+		next[colIdx] = dimName;
+		const url = new URL(page.url);
+		url.searchParams.set('metrics', next.filter((n): n is string => Boolean(n)).join(','));


+				<div class="SegmentedControl" role="tablist" aria-label="Result type">
+					<button
+						type="button"
+						role="tab"
+						aria-selected={activeKind === 'prompts'}


+					<button
+						type="button"
+						role="tab"
+						aria-selected={activeKind === 'scenarios'}


+let runColor = $derived(
+	Object.fromEntries(data.runs.map((r, i) => [r.run_id, RUN_COLORS[i]])) as Record<string, string>
+);


* docs: normalize commands and path separators * fix(viewer): truncate long callable target labels in compare view (#160) The compare page derived a short label from `run.model` by splitting on `/` only (lines 301, 376). When the target is a Python callable like `examples.bank_manager_demo.agent:chat_unguarded`, that returned the entire dotted path and overflowed the per-run card body (line 211) plus the "By behavior category" column headers — two adjacent header cells visibly crashed into each other. Adds a small `runLabel` helper that splits on '':'' first (callable targets) then `/` (provider/model paths). The card body now wraps the short label in a `truncate + title` tooltip so the full path is still visible on hover. Splits off the label-fix portion of the original PR #78 (which also attempted a 3-way URL fix that's now stale). The 3-way URL plumbing is out of scope here and can ship in a separate PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * init: fix interview flow (context vs system_prompt, judge dimensions, default model) + consolidate YAML emission rules (#176) * init: add --default-model CLI flag and surface design-agent model to LLM - New --default-model option on `assert-eval init` lets the user pre-seed the pipeline.default_model hint for the interview. - _build_default_model_hint() emits a system-prompt section when the hint is provided. - run_design_loop() always tells the LLM which model is driving the design conversation, and conditionally surfaces the default_model hint as the first user-visible message. This is the plumbing the prompt update relies on; no behavior change without the prompt edit that follows. * prompt(init): separate context from system_prompt, restructure judge, require default_model - Section 1 renamed 'Application Context' and explicitly distinguishes developer narrative from target.system_prompt for hosted-model targets. - New Section 3 'Pipeline Default Model' rules: never silently copy the target model into default_model; ask explicitly. - Renumbered Behavior (4), Test Set (5), Judge (6). - Judge built-ins: policy_violation, overrefusal. - Custom judge dimensions now use ONE consolidated turn for name + description + rubric-true + rubric-false (instead of 4 separate turns). - Added explicit guideline against conflating context with target.system_prompt. * tests(init): cover --default-model plumbing and prompt anchors - test_prompt_contains_required_section_anchors: low-resolution canary that guards against accidental deletion of the new sections during future prompt refactors. Not a behavior test — its docstring tells future contributors they can reword freely as long as anchors stay. - test_prompt_includes_default_model_hint_when_provided: exercises _build_default_model_hint() end-to-end via build_system_message(). - test_design_agent_surfaces_model_hint_to_llm: CLI->design loop integration test asserting the first user message names both the design-agent model and the --default-model hint. * prompt(init): comment out per-stage model examples in schema reference Schema-reference YAML examples in init_system.md were showing live 'model:' blocks under systematize, test_set.prompt, test_set.scenario, inference.tester, and judge. The design agent treats these as copy-paste templates and emits them as live YAML, which then overrides default_model silently. Comment out every per-stage 'model:' example in the schema, add an explicit Section 3 prohibition on emitting uncommented per-stage 'model:' blocks unless the user explicitly asked for an override, and update the Discoverable defaults guideline to require commented-only surfacing of per-stage overrides. * prompt(init): fix tester-toggle guideline to use commented model example The tester-toggle Guidelines bullet was the last place in the prompt showing a live, uncommented per-stage 'model:' block. Even with Section 3 prohibiting live per-stage model overrides, the design agent still copied this bullet's YAML verbatim into proposals, causing 'tester:\n model:\n name: ...' to leak into generated configs. Bring the bullet in line with the schema-reference examples by commenting out the model override and keeping only the bare 'tester:' key live. * prompt(init): consolidate YAML emission rules into one section The per-stage 'model:' rule, the tester-toggle, target.trace, and '# customize:' / '# review:' conventions were each repeated and lightly contradicted across Section 3, the schema reference, and the Guidelines list. The LLM kept rediscovering uncommented per-stage 'model:' templates because the rule had no single home. Consolidate all YAML emission rules into a new top-level '# YAML emission rules' section between '# Config Structure' and '# Guidelines' with four sub-sections (per-stage models, tester block, target.trace, customization hints). Trim Section 3 to the ask-phase conversation flow only and cross-reference the new section. Drop the duplicated tester/target.trace/customization bullets from Guidelines. Also two cosmetic fixes that surfaced during the audit: - Section 'Pacing' said 'all 5 sections' but there are 6 ask sections. - 'Test Set Dimensions' was at heading level 5 (#####) while everything else at that depth uses level 4 (####). * rename: assert_eval -> assert_ai (module dir + pyproject) Renames the package directory and updates the five distribution-name and module-name references in pyproject.toml (name, [project.scripts], all extra, dev dependency-group, and setuptools packages.find). Also drops examples* from the wheel per the PyPI publishing plan. Imports inside the package are not yet updated -- follow-up commit. * rename: update intra-package imports to assert_ai Replaces assert_eval / assert-eval references inside the moved package so it imports cleanly. Covers from/import statements, dotted module strings (telemetry tags, error messages), and internal task/thread names (assert-eval-watchdog -> assert-ai-watchdog, etc.). Tests, docs, scripts, and CI updated separately. * rename: update tests for assert_ai Replaces assert_eval / assert-eval references in tests/ — module patches, dotted import paths, CLI invocations, and any expected log/identifier strings. * rename: update top-level assert_ai modules Catches the depth-1 .py files in assert_ai/ that the package-imports commit missed: cli.py, config.py, display.py, results.py, runner.py, viewer_read_model.py. Same dual-pattern (assert_eval -> assert_ai, assert-eval -> assert-ai). * rename: update docs, examples, scripts, viewer, and website Sweeps assert_eval -> assert_ai and assert-eval -> assert-ai across user-facing surfaces: root README and AGENTS, docs/ tree, runnable examples (including notebooks and agent scripts), helper scripts, the SvelteKit viewer, the marketing website, and the analysis README that ships inside the wheel. * rename: update CI workflow path filter Path filter in .github/workflows/regression.yml now points at assert_ai/** instead of assert_eval/**. * rename: update .gitignore policy artifact pattern Renames assert_eval_policy.* -> assert_ai_policy.* so runtime policy artifacts stay ignored after the package rename. * chore: rename root package in uv.lock to assert-ai Hand-edits the 4 self-references to the root project in uv.lock to match the renamed distribution. uv lock cannot regenerate cleanly today because of a pre-existing dspy-ai>=2.7,<3 vs >=3 conflict on main; a full lockfile refresh will land separately once that is resolved. * docs: replace stale microsoft/adaptive-eval URLs with responsibleai/ASSERT (#178) The microsoft/adaptive-eval slug 404s after the rename to ASSERT. Two customer-facing docs still pointed at the dead URL: - CONTRIBUTING.md: dev-setup clone snippets (bash + powershell) - AGENTS.md: paste-in prompt block that downstream LLMs hand to users Both now point at https://github.com/responsibleai/ASSERT. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(website): use hyphenated assert-ai CLI in terminal demo The CLI entry point in pyproject.toml is 'assert-ai' (hyphen), but the website's terminal typing animation showed 'assert_ai run', which would mislead users copying the command. Reported by Copilot review on PR #177. * build: restore examples* in packages.find include The previous include list was ["assert_eval*", "examples*"]; the rename commit accidentally dropped 'examples*'. Tests import from 'examples.agents.*' (test_tool_module_sandbox, test_openclaw_driver), and the editable/wheel install needs to continue exposing the examples package to preserve current behavior. Packaging cleanup (e.g. removing examples from the published wheel) will be handled in a follow-up PR, not as part of this mechanical rename. Reported by Copilot review on PR #177. * docs(assets): rename framework diagram to assert-ai-framework-diagram.png README.md was updated to reference assets/assert-ai-framework-diagram.png but the asset file itself was not renamed, breaking the image in the rendered README. Reported by Copilot review on PR #177. * Website: partner quotes section, expanded framework logo loop, sidebar polish * refactor(prompts): move internal-pipeline-prompts under assert_ai subpackage Move the prompt template directory inside the package so it ships inside the wheel as data alongside the importable code. Pure file move plus an empty __init__.py so importlib.resources can discover it as a real subpackage. The next commit switches io.py to load via importlib.resources; reads from the old top-level directory will start failing after that change. * fix(io): resolve PROMPTS_DIR via importlib.resources The previous implementation derived the prompts directory from `Path(__file__).resolve().parents[2]`, which works in a repo checkout but lands inside `site-packages/` for a wheel install — where the top-level `internal-pipeline-prompts/` directory does not exist. Switch to `importlib.resources.files('assert_ai.internal_pipeline_prompts')` so resolution works both from source and from an installed wheel. The returned Traversable still supports `/`, `.read_text()`, `.read_bytes()`, and `.is_file()`, which is everything the existing call sites need. `artifact_cache._prompt_descriptor` is updated to use `.is_file()` instead of `.exists()` since Traversable does not guarantee `.exists()`. * build: drop examples* from packages.find include `examples*` was previously included in `packages.find`, which made the wheel ship every example agent and bloat the distribution. The examples are not importable Python packages from `assert_ai` — they are standalone walkthroughs that users run from a repo checkout. They should not be in the wheel. * build(pyproject): add license, authors, keywords, classifiers, urls, package-data Adds the metadata PyPI surfaces on the project page and uses for discovery / filtering, plus the package-data declarations needed to ship the prompt templates inside the wheel: - `license = { file = 'LICENSE' }` (MIT, already at repo root) - `authors` with the team display name; email is left as a TODO until a public contact alias is available (does not block publishing) - `keywords` (safety, evaluation, llm, agent, responsible-ai) - `classifiers` covering Development Status (Beta), MIT license, Python 3.11/3.12/3.13, and AI / QA / Testing topics - `[project.urls]` (Homepage, Repository, Issues, Documentation) - `[tool.setuptools] include-package-data = true` so any future data files in tracked packages ship in the wheel - `[tool.setuptools.package-data]` explicitly listing `assert_ai.internal_pipeline_prompts = ['*.md']`. Without this, `include-package-data` alone is a no-op for plain setuptools (no MANIFEST.in, no setuptools-scm), and the .md prompt files would be excluded from the wheel — silently re-introducing the bug the previous commit fixes. * ci: add build.yml workflow (PEP 517 build + cross-platform install smoke) `build.yml` runs on every push to main, every PR to main, and on `workflow_dispatch`. Two jobs: 1. `build` (ubuntu-latest, Python 3.11): - `python -m build` (PEP 517 sdist + wheel) - `python -m twine check dist/*` (verifies long-description renders for PyPI) - uploads `dist/` as a workflow artifact with conditional retention: 14 days for PR builds, 90 days for main and dispatch builds so a merged commit's wheel stays available for downstream consumers. 2. `test-install` (3x3 matrix: ubuntu / macos / windows x Python 3.11 / 3.12 / 3.13, fail-fast off): - downloads the wheel artifact - installs it into a fresh environment - runs `assert-ai --help` to prove the entry point resolves and the package + bundled prompts import successfully. This is the runtime regression net for the wheel-install bug fixed in the previous commit, and it runs purely against the built wheel (not the repo checkout) so any `Path(__file__).parents` style regression will fail the matrix instead of slipping into a release. `permissions: contents: read` only - this workflow never writes back to the repo and never talks to PyPI. * chore(env): rename ASSERT_EVAL_* env vars to ASSERT_AI_* Clean break to match the package name (assert-ai). After the package was renamed from assert-eval to assert-ai, users would naturally reach for ASSERT_AI_* env vars and silently get nothing under the old prefix. Renamed (no compatibility shim): - assert_ai/cli.py: Click auto_envvar_prefix ASSERT_EVAL -> ASSERT_AI (so e.g. ASSERT_AI_CONFIG=... wires the --config flag) - viewer/src/lib/server/run-spawn.ts: ASSERT_EVAL_COMMAND override and its log/error strings -> ASSERT_AI_COMMAND - examples/science_research_agent/tools.py and README.md: ASSERT_EVAL_REAL_TOOLS_NOCACHE -> ASSERT_AI_REAL_TOOLS_NOCACHE Follow-up to PR review feedback on the package rename PR. * docs(examples): fix stale P2M_* env-var names in READMEs to match code The example agents read ASSERT_AZURE_DEPLOYMENT and ASSERT_TARGET_MODEL, but three READMEs still documented the legacy P2M_* names from before the package rename. That left users setting the wrong variable and silently getting the default model. Aligns README docs with the code: - examples/travel_planner_langgraph/README.md: P2M_AZURE_DEPLOYMENT -> ASSERT_AZURE_DEPLOYMENT (inline comment + var table) - examples/phoenix_auto_trace/README.md: P2M_AZURE_DEPLOYMENT -> ASSERT_AZURE_DEPLOYMENT - examples/travel_planner_neurosan/README.md: P2M_TARGET_MODEL -> ASSERT_TARGET_MODEL Docs-only; no behavior change. tests/test_no_p2m_references.py still passes (the guard uses \bp2m\b which doesn't catch P2M_* tokens). * docs: add CHANGELOG.md + README migration note for assert_ai rename (#185) Documents the assert_eval -> assert_ai package/CLI rename (PR #177) and the ASSERT_EVAL_* -> ASSERT_AI_* env var rename (PR #182) so existing preview users have a clear migration path. Keep a Changelog 1.1.0 format for CHANGELOG.md. README gets a short [!IMPORTANT] callout near the top. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(viewer): rename spawnAssertEvalRun / resolveAssertEvalCommand to AssertAi* (#187) Catches up TS symbol names with PR #177 (assert_eval -> assert_ai package rename) and PR #182 (ASSERT_EVAL_* -> ASSERT_AI_* env var rename). The viewer function names still carried the old prefix - this completes the rename across the TS surface. No behavior change. Runs npm run check clean (same 3 pre-existing errors as documented in PR #180; zero new errors). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: add community-launch hygiene (issue/PR templates, CODEOWNERS, badges) (#186) Bundle D for public-preview readiness. Adds: - .github/ISSUE_TEMPLATE/{bug_report.yml,feature_request.yml,config.yml}: form-style issue templates with required-field validation, secret-redaction reminders, and discussions link in config.yml. - .github/PULL_REQUEST_TEMPLATE.md: short PR template with summary, motivation, testing notes, and a brief checklist. - CODEOWNERS: placeholder catch-all rule pointing to @responsibleai/assert-maintainers (Chang to update team handle). - README badges: CI build status (from PR #182's build.yml workflow), supported Python versions (3.11/3.12/3.13 matching CI matrix), license. PyPI badge intentionally omitted until the package is published. No code change. No CHANGELOG.md (parallel PR delivers that). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(prompts): rename "seed" -> "test case" in internal_pipeline_prompts/ (#188) Customer-facing terminology cleanup (M-3 from prior audit). The renames in PR #181 originally targeted `internal-pipeline-prompts/` but those edits had to be dropped during rebase since PR #182 git mv-ed the directory under `assert_ai/internal_pipeline_prompts/`. Re-applying against the new path. Scope: prompt markdown files only. Python code comments retain internal terminology per project convention. One occurrence intentionally preserved ("seed config via --from" in init_system.md L436 — refers to a starter config, not a test case). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs+examples: post-rename cleanup, jargon scrub, broken nav fixes (#181) * tests: tighten p2m guard regex and skip binary/lockfile false positives The previous \\b p2m \b\ word-boundary regex missed env-var leaks like P2M_AZURE_DEPLOYMENT because the underscore broke the word boundary. Switched to a case-insensitive substring match so P2M_* env vars and other prose leaks are caught. To keep the test signal clean, also skip: - Binary file extensions (.svg, image formats) that may embed base64. - Lockfiles (package-lock.json, yarn.lock, pnpm-lock.yaml, uv.lock, poetry.lock) where sha512 hashes coincidentally contain 'p2m'. Added a sanity test that asserts the regex catches P2M_AZURE_DEPLOYMENT. Deleted an orphan website/public/icons/P2M Thumbnail.svg (no references). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * examples/docs: replace internal model names with public substitutes Replaced internal-only model names with publicly available substitutes across examples, docs, and the website snippets so customers can run copy-paste configs without hitting unknown-model errors. - azure/gpt-5.4-mini -> azure/gpt-4o-mini - azure/gpt-5.4-nano -> azure/gpt-4o-mini - azure/gpt-5.4 -> azure/gpt-4o - GPT-5-nano (prose) -> gpt-4o-mini - GPT-5-railfree references removed (publishable substitute does not exist) Scope deliberately excludes assert_ai/ core code, tests/, .github/ workflows, and artifacts/results/** (frozen historical records). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * AGENTS.md: complete rename to ASSERT (branding + dead doc links) PR #178 fixed the GitHub URL but the prose, title, and doc cross-links still referenced 'Adaptive Eval' and pointed to renamed/moved doc paths. - AGENTS.md title and prose: 'Adaptive Eval' -> 'ASSERT' - Dead doc links updated: - docs/quickstart.md -> docs/getting-started.md - docs/writing-eval-specs.md -> docs/guides/create-evaluation.md - docs/reading-results.md -> docs/guides/results.md - Same branding fix applied to .cursorrules, .devcontainer/devcontainer.json, CONTRIBUTING.md, SUPPORT.md, and examples/azure_doc_qa/IMPROVEMENT_JOURNEY.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * examples/change_control_agent: genericize internal Microsoft infra references The change_control_agent example referenced internal Microsoft deployment/ops systems (Safefly, Ev2, R2D, ICM, ADO, ChangeKeep) which are unfamiliar to external customers and leak internal context. Renamed to generic enterprise vocabulary so the example reads as a generic change-control pattern: - submit_to_safefly -> submit_to_deployment_gateway - submit_to_ev2 -> submit_to_rollout_service - submit_to_r2d -> submit_to_release_readiness - create_ado_change_request -> create_change_request - get_icm_incident -> get_incident - ChangeKeep -> ChangeFlow - SAFEFLY-<id> -> DEPLOYGATE-<id> - 'Safefly', 'Ev2', 'R2D' -> 'Deployment Gateway', 'Rollout Service', 'Release Readiness' - 'ADO', 'ICM' -> 'change-tracker', 'incident-tracker' - 'internal change-management assistant' -> 'change-management assistant' Both function names and the string-literal handles used as dict keys/ identifiers in tools.py were renamed so the example remains functional. README updated to match. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * examples/incident_triage_agent: rewrite README opening for customer context The README opening framed this example as an 'ACS efficacy demo' with an 'A -> C demo path', cross-linked an internal draft PR (#88), and referenced a bank-manager demo that does not ship in this repo. Rewrote the title, opening paragraph, and TL;DR to present the example as a generic incident-triage agent evaluation. Preserved the variant tables, eval-config matrix, and provenance section unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * examples/README: fix broken paths, add missing example entries - Quickstart YAML path: pipes/ -> prompt_agents/ (the pipes/ directory was renamed long ago; the README pointed at a 404). - Added table rows for four examples that exist on disk but were not listed: travel_planner_neurosan, change_control_agent, azure_doc_qa, benchmark. - Fixed dead doc links: docs/reference/cli.md -> docs/cli/commands.md. - Removed broken link to docs/case-study-incident-triage-joint.md (file does not exist). - Updated the layout block to match the current directory structure. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * README: add inline Quick install block Surface a copy-pasteable install/run snippet above the 4-column 'Get started' table so first-time readers don't have to click through to find the bootstrap commands. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * scripts + pyproject: misc jargon and dead-reference cleanup - scripts/migrate_artifacts_to_pr23_vocab.py: docstring no longer references 'PR #23 vocabulary'; describes the migration in customer- neutral terms. - pyproject.toml: trimmed dspy pin comment to drop the bank-manager reference (that demo does not ship in this repo). - scripts/README.md, scripts/scenario_failure_prediction.py: replaced the relationship-entanglement-v1 suite name with the placeholder <your-suite-name> so examples are generic. - scripts/judge_stability_experiment.py: dropped internal model name. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: clean-slate the public-facing surface — drop private-preview migration callouts (#189) The repo should read like a clean v1 launch, not a project mid-migration. Private-preview era transitions (assert_eval -> assert_ai, ASSERT_EVAL_* -> ASSERT_AI_*, prompts directory move) are noise in public docs because there were no public releases to break. Existing private-preview users are notified out-of-band. - README.md: drops the [!IMPORTANT] migration callout from PR #185. Added a bundled-viewer bullet to the capability list. - CHANGELOG.md: rewritten to empty [Unreleased] scaffold (Keep a Changelog 1.1.0), ready to populate when v0.1 ships. - .github templates: replaced stale `assert-eval` placeholders with `assert-ai` so contributor-facing examples match the current CLI. No code change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(tests): make test_runtime_safety subprocess inherit parent sys.path after assert_ai rename (#190) The subprocess invocation in tests/test_runtime_safety.py::test_run_stage_coro_does_not_block_subprocess_exit_when_worker_leaked spawned a fresh sys.executable but did not forward the parent interpreter's import paths. In CI, pip install -e . populates site-packages so import assert_ai works inside the subprocess; in any environment where pytest is the only thing putting the project root on sys.path (developer running pytest without first installing, or a leftover venv from before the rename), the subprocess hits ModuleNotFoundError: No module named 'assert_ai' and the test fails before it can even exercise the leaked-worker shutdown path. Fix: build PYTHONPATH from the parent's sys.path and pass it via env= to subprocess.run. Works whether assert_ai is editable-installed or only discovered through pytest's rootdir hook. Pre-existing failure since PR #177 (assert_eval -> assert_ai rename) — the import statement was correctly renamed but the underlying env-propagation gap was unmasked once the package name no longer matched any stale install left in dev venvs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: rename CLI/python refs and clean user-facing wording * docs: refine docs index and migration wording * Clean up of migration terminology and doc updates * docs: rename CLI/python refs and clean user-facing wording * docs: refine docs index and migration wording * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Restore init example context in getting started docs * examples(bank-manager): ACS-vs-unguarded comparison demo Self-contained example comparing an unguarded LangGraph bank-manager agent against the same agent guarded by the new Agent Control Specification (ACS) runtime. Two ASSERT callables, two eval configs, two frozen n=100 result snapshots for viewer playback. Contents (scoped to examples/bank_manager_agent_shield/): - agent.py: chat_unguarded + chat_guarded_acs - mcp_server.py: mock banking MCP server - acs/manifest.yaml + acs/policy/bank_manager.rego: stateless ACS policy (SSN input, sensitivity-scoped read/transfer gates, approval/admin-mode gates, prompt-injection scrubber) - eval_unguarded_v2.yaml: baseline (owns systematize+test_set) - eval_guarded_acs.yaml: ACS variant (reuses baseline test_set) - results/: frozen n=100 artifacts for both variants ACS integration uses the idiomatic SDK orchestration helpers: control.run() for input/output gating around the agent execution, control.run_tool() per MCP tool for pre/post tool-call gating. Per-turn state (transfer_approved, admin_mode_active, account_sensitivity) is tracked by the host wrapper and threaded into each snapshot, since ACS is stateless by design. Headline n=100 (same test set across both variants): unguarded: safety_violation 39%, unjustified_refusal 2% ACS: safety_violation 9%, unjustified_refusal 2% README documents both usage paths: seed the committed results into artifacts/results/ and view them, or run the full pipeline end-to-end (requires agent_control_specification SDK + opa binary on PATH). * examples(bank-manager): rename judge dims to policy_violation/overrefusal Re-judged the existing n=100 inference outputs against the renamed rubric (no inference re-run). Updated taxonomy.json behavior_categories to match the names referenced by the committed test_set.jsonl so the viewer renders. Regenerated .viewer/ caches. Results (n=100, byte-identical inference): unguarded: policy_violation 40%, overrefusal 4% ACS: policy_violation 5%, overrefusal 15% * chore(bank-manager-acs): rename eval_unguarded_v2.yaml to eval_unguarded.yaml The _v2 suffix is no longer meaningful; drop it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(bank-manager-acs): drop 4 target_error scenarios from unguarded variant The unguarded variant-a-unguarded-n100 artifacts contained 4 scenario rows that failed with stop_reason="target_error": test_case_000053, test_case_000063, test_case_000065, test_case_000090 Root cause: examples/bank_manager_agent_shield/agent.py:309 calls asyncio.run(_run_agent_async(...)) inside a worker thread. asyncio.run creates a new event loop and refuses to nest, so under concurrent scenario load the TaskGroup unwinds with an unhandled-errors crash. These are inference errors, NOT ACS policy blocks. The ACS-guarded variant ran clean at n=100. This commit drops the 4 affected rows from every downstream artifact in variant-a-unguarded-n100/ so the headline numbers reflect only the 96 cases that actually ran: - inference_set.jsonl: 100 -> 96 rows - scores.jsonl: 100 -> 96 rows - .viewer/viewer_audit_rows.json: 50 -> 46 (the 4 were scenarios) - .viewer/viewer_prompt_rows.json: 50 -> 50 (unaffected) - .viewer/viewer_score_index.json: byte offsets rebuilt - .viewer/viewer_transcript_index.json: byte offsets rebuilt - .viewer/viewer_run_manifest.json: source_files size_bytes patched - metrics.json: stages.judge.calls 100 -> 96 (skipped rows weren't actually judged) Recomputed unguarded headline from the surviving 96 scored rows: policy_violation 42% (was 39%), overrefusal 4% (was 2%). README updated to reflect n=96 for unguarded and n=100 for ACS-guarded. Follow-up suggestion: fix chat_unguarded / chat_guarded_acs to not call asyncio.run from within a thread. Either restructure as a coroutine and let the caller await it, or use a long-lived loop via asyncio.new_event_loop()/loop.run_until_complete() guarded by a lock. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore(bank-manager-acs): rename bank_manager_agent_shield to bank_manager_agent_control Matches the broader ACS = Agent Control Specification framing. Folder rename is via git mv to preserve blame. All in-repo references (Python imports, YAML targets, frozen artifact configs, viewer rows, README, scripts/render_trade_off.py) updated: - 712 underscore-form occurrences (bank_manager_agent_shield) - 12 dash-form occurrences (bank-manager-agent-shield) — suite IDs inside taxonomy.json, config.yaml, and README copy commands Internal artifact directory names (variant-a-unguarded-n100, variant-e-guarded-acs-n100) are unchanged — those are run names, not suite names. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(bank-manager-acs): remap test_set behaviors to formal taxonomy names The systematize stage regenerated taxonomy.json with 4 formal category names but test_set.jsonl + downstream artifacts retained the older descriptive names, causing the viewer to 500 at viewer/src/lib/server/data.ts:184 (behaviorDefinition strict lookup). Maps every test case's dimensions.behavior + top-level behavior fields to the canonical formal name across: - examples/bank_manager_agent_control/results/test_set.jsonl - per-variant inference_set.jsonl, scores.jsonl, .viewer/*.json Mapping: authorized banking tool use -> Authorization-gated action handling accurate financial statements ... -> Correct tool use and customer-facing banking help confidentiality and social-engineering -> split per-case into Confidential data withholding OR Resistance to impersonation and prompt injection Each of the 4 formal categories now has >= 1 test case (the 4th was empty before). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(bank-manager-acs): rename assert-eval -> assert-ai in demo README Catches up the demo README to the post-rebrand CLI binary name. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore(bank-manager-acs): catch up demo to post-rebrand naming (assert-ai, ASSERT_AI_*) After rebasing onto main, sweeps the demo for stale terminology: - p2m run -> assert-ai run (in YAML comments + README) - p2m/stages/* -> assert_ai/stages/* (in YAML comments) - ASSERT_EVAL_* -> ASSERT_AI_* (env vars) - assert_eval -> assert_ai (module paths, if any) - assert-eval -> assert-ai (CLI binary) - drops references to eval_guarded_v2.yaml / eval_guarded_v3.yaml (now eval_guarded_acs.yaml only) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(bank-manager-acs): add Phoenix 2-line auto-instrumentation to agent.py Drops in the canonical Phoenix auto_instrument pattern at the top of agent.py so LangChain / OpenAI / MCP tool calls flow into Phoenix without any framework config. Optional via try/except — demo still runs without arize-phoenix-otel. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(viewer/metrics): suppress permissibility-split cards when taxonomy is single-sided The "Permissible requests failed" / "Not-permissible requests failed" card pair only makes sense when the taxonomy has BOTH permissible and not-permissible behaviors. For single-sided taxonomies the not-aligned bucket would render an empty "no relevant judgments" tile that's noise. Now returns null/null when the permissibility index has < 2 distinct values, so the run-detail page's `policyViolationOnPermissible || policyViolationOnNotPermissible` gate naturally skips the section. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(bank-manager-acs): add variant-c prompt-engineering intervention Adds chat_unguarded_prompted callable that runs the same raw LangGraph agent as chat_unguarded but with a defensive addendum appended to the system prompt (no tool gating). Reuses the suite-root frozen test_set + same judge dims for a fair 3-way comparison: bare baseline (a) vs prompt-engineering (c) vs ACS-guarded (e). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Minsoo Thigpen <mithigpe@microsoft.com> Co-authored-by: changliu2 <99364750+changliu2@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: tangym <tangym@users.noreply.github.com> Co-authored-by: sooyeonni <29706402+sooyeonni@users.noreply.github.com> Co-authored-by: sooyeonni <dusl1209@naver.com> Co-authored-by: Minsoo Thigpen <minsoo.thigpen@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Mike Shi <peichengshi@microsoft.com> Co-authored-by: Chang Liu <changliu2@microsoft.com>

changliu2 and others added 4 commits June 1, 2026 00:18

changliu2 mentioned this pull request Jun 1, 2026

refactor(viewer): rename spawnAssertEvalRun / resolveAssertEvalCommand to AssertAi* #187

Merged

changliu2 and others added 2 commits June 1, 2026 02:47

Copilot AI mentioned this pull request Jun 1, 2026

docs: metrics.json is token-usage telemetry, not aggregate eval rates — no rename #192

Merged

changliu2 and others added 4 commits June 1, 2026 02:18

tangym requested a review from Copilot June 1, 2026 23:11

Copilot started reviewing on behalf of tangym June 1, 2026 23:11 View session

Copilot stopped reviewing on behalf of tangym due to an error June 1, 2026 23:11
The job was not started because recent GitHub Actions payments have failed or your spending limit needs to be increased.

Copilot AI reviewed Jun 1, 2026

View reviewed changes

tangym approved these changes Jun 1, 2026

View reviewed changes

changliu2 merged commit fc6f210 into main Jun 1, 2026
10 of 11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

viewer/compare: pin baseline left + show only populated dim columns - #180

viewer/compare: pin baseline left + show only populated dim columns#180
changliu2 merged 10 commits into
mainfrom
changliu2/viewer-compare-baseline-pin-and-populated-dims

changliu2 commented Jun 1, 2026

Uh oh!

changliu2 commented Jun 1, 2026

Uh oh!

changliu2 commented Jun 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

changliu2 commented Jun 1, 2026

viewer/compare: pin baseline left + show only populated dim columns

TL;DR

Decisions worth flagging

How to verify

CI / typecheck

Out of scope

Uh oh!

changliu2 commented Jun 1, 2026

Uh oh!

changliu2 commented Jun 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants