Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/purple-pets-fail.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"@hyperdx/common-utils": patch
"@hyperdx/api": patch
"@hyperdx/app": patch
---

feat: Add source scoping to dashboard filters
10 changes: 10 additions & 0 deletions packages/api/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -2349,6 +2349,16 @@
"description": "Language of the where condition",
"default": "sql",
"example": "lucene"
},
"appliesToSourceIds": {
"type": "array",
"items": {
"type": "string"
},
"description": "Optional list of source IDs this filter applies to. Omit or provide\nan empty array to apply the filter to ALL tiles regardless of source.\nA non-empty array restricts the filter to only tiles whose source ID\nis in the list; tiles using other sources are not affected by the\nselected filter value(s).\n",
"example": [
"65f5e4a3b9e77c001a111111"
]
}
}
},
Expand Down
36 changes: 24 additions & 12 deletions packages/api/src/mcp/prompts/dashboards/content.ts
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ Apply these before calling hyperdx_save_dashboard. Each rule is enforced by the

8. HEATMAPS FOR DISTRIBUTIONS. Trace duration buckets, payload size buckets. Trace sources only. Set numberFormat: { output: "duration", factor: 0.000000001 } on the chart config so the y-axis reads in human time.

9. FOR FOCUSED PER-DIMENSION DASHBOARDS, DECLARE A DASHBOARD-LEVEL FILTER. Pass filters: [{ type: "QUERY_EXPRESSION", name, expression, sourceId }] at the top level. The user gets a dropdown in the dashboard header; every tile on the same source re-scopes when a value is picked. Do NOT hardcode the dimension into each tile's where clause.
9. FOR FOCUSED PER-DIMENSION DASHBOARDS, DECLARE A DASHBOARD-LEVEL FILTER. Pass filters: [{ type: "QUERY_EXPRESSION", name, expression, sourceId }] at the top level. The user gets a dropdown in the dashboard header; by default every tile is re-scoped when a value is picked. On mixed-source dashboards add appliesToSourceIds: ["<id>", ...] to restrict the filter to only the tiles whose source carries that column. Omit the field to keep the broadcast-to-all-tiles default. Do NOT hardcode the dimension into each tile's where clause.

10. UPDATE IS REPLACE, NOT MERGE. hyperdx_save_dashboard with an id overwrites tiles, containers, and filters in their entirety. Call hyperdx_get_dashboard first when you only want to add or rename one entry; do not send a partial set or you will silently drop everything you omitted.

Expand Down Expand Up @@ -958,24 +958,36 @@ to plot the first as a ratio of the second. Useful for error rates:

== DASHBOARD FILTERS ==

Optional dashboard-level filter declarations. Each entry adds a dropdown to the dashboard header that scopes every tile against the same source. Use this for focused per-dimension dashboards (per-service, per-tenant, per-endpoint) instead of hardcoding the dimension into every tile's where clause.
Optional dashboard-level filter declarations. Each entry adds a dropdown to the dashboard header that scopes tiles against the filter's expression. Use this for focused per-dimension dashboards (per-service, per-tenant, per-endpoint) instead of hardcoding the dimension into every tile's where clause.

Filter shape:
{ type, name, expression, sourceId, where?, whereLanguage? }
{ type, name, expression, sourceId, where?, whereLanguage?, appliesToSourceIds? }

type "QUERY_EXPRESSION" (the only currently supported type).
name Human label shown in the filter dropdown (e.g. "Service").
expression Column or attribute path the filter scopes (e.g. "ServiceName" or "SpanAttributes['tenant.id']").
sourceId Which source the expression resolves against. Tiles on a different source are NOT scoped by the filter.
where Optional pre-filter that narrows the set of distinct values offered in the dropdown.
whereLanguage "lucene" or "sql". Defaults to "lucene".
type "QUERY_EXPRESSION" (the only currently supported type).
name Human label shown in the filter dropdown (e.g. "Service").
expression Column or attribute path the filter scopes (e.g. "ServiceName" or "SpanAttributes['tenant.id']").
sourceId Which source the dropdown VALUES are queried from. Independent of which tiles get filtered (see appliesToSourceIds below).
where Optional pre-filter that narrows the set of distinct values offered in the dropdown.
whereLanguage "lucene" or "sql". Defaults to "lucene".
appliesToSourceIds Optional list of source IDs the filter applies to. Omit (or pass undefined) to apply the filter to EVERY tile regardless of source (the recommended default). Pass a non-empty array to restrict the filter to tiles whose source is in that list, useful on mixed-source dashboards where the column only exists on some sources.

Example:
Example (broadcast to every tile, the common case):
filters: [
{ type: "QUERY_EXPRESSION", name: "Service", expression: "ServiceName", sourceId: "<trace-source-id>" }
]

When a value is picked in the dropdown, the renderer combines it with each tile's existing where clause via AND. Tiles do NOT need to reference the filter name; the source match alone is enough.
Example (scoped to the trace source only on a mixed log/trace/metric dashboard):
filters: [
{
type: "QUERY_EXPRESSION",
name: "Service",
expression: "SpanName",
sourceId: "<trace-source-id>",
appliesToSourceIds: ["<trace-source-id>"]
}
]

When a value is picked in the dropdown, the renderer combines it with each in-scope tile's existing where clause via AND. Tiles do NOT need to reference the filter name; matching the scope (or no scope set) is enough.

== TABLE TILE LINKING (config.onClick) ==

Expand Down Expand Up @@ -1197,7 +1209,7 @@ Example: find top patterns for production services over the last 4 hours:
7. Hardcoding a focus dimension into every tile's where clause
Wrong: five tiles, each with where: "ServiceName:checkout"
Correct: filters: [{ type: "QUERY_EXPRESSION", name: "Service", expression: "ServiceName", sourceId: "<id>" }]
The dashboard filter applies globally; tiles do not need the literal.
The dashboard filter applies globally; tiles do not need the literal. On mixed-source dashboards where only some tiles carry the column, add appliesToSourceIds: ["<id>", ...] to scope the filter to just those sources instead of breaking the unrelated tiles.

8. Forgetting to validate tiles after saving
Always call hyperdx_query_tile on EVERY tile after hyperdx_save_dashboard, not just one. Save validates input shape but not query semantics. Several known gaps (Lucene comparison/wildcard on map attributes, builder tiles on metric sources, malformed having) pass save and fail at render time. A dashboard with one bad tile renders the whole page in a degraded state; the user sees "Error loading chart" with no way to know which tile broke unless you validated. If query_tile returns an error, fix the where / SQL and re-save before declaring the dashboard ready.
Expand Down
24 changes: 23 additions & 1 deletion packages/api/src/mcp/tools/dashboards/schemas.ts
Original file line number Diff line number Diff line change
Expand Up @@ -688,6 +688,18 @@ const mcpDashboardFilterSchema = z
whereLanguage: SearchConditionLanguageSchema.describe(
'Filter language for `where` ("lucene" or "sql"). Optional, but set it explicitly.',
),
appliesToSourceIds: z
.array(objectIdSchema)
.optional()
.describe(
'Optional list of source IDs that this filter is applied to. ' +
'Omit (or pass `undefined`) to apply the filter to ALL tiles regardless of source ' +
'— this is the recommended default. ' +
'A non-empty array restricts the filter to only tiles whose source ID is in the list; ' +
'tiles on other sources are not affected by the dropdown value at all. ' +
'Useful on mixed-source dashboards where a column (e.g. SpanName) only exists on ' +
'a subset of sources.',
),
})
.describe(
'A dashboard-level filter the user can adjust in the dashboard filter bar. ' +
Expand All @@ -704,10 +716,20 @@ export const mcpFiltersParam = z
'If another tile\'s onClick targets THIS dashboard with `filters: [{ expression: "X", ... }]`, ' +
'this array MUST declare a filter whose `expression` is "X". Otherwise the value is ' +
'dropped on arrival and the destination opens unfiltered.\n\n' +
'Example:\n' +
'By default a filter applies to every tile on the dashboard. On mixed-source dashboards, ' +
'use the optional `appliesToSourceIds` field to restrict a filter to only the tiles whose ' +
'source carries the referenced column — leave `appliesToSourceIds` omitted to keep the ' +
'broadcast-to-all-tiles default.\n\n' +
'Example (broadcast to every tile):\n' +
'[\n' +
' { "type": "QUERY_EXPRESSION", "name": "Service", "expression": "ServiceName",\n' +
' "sourceId": "<trace-source-id>", "whereLanguage": "sql" }\n' +
']\n\n' +
'Example (scoped on a mixed log/trace/metric dashboard):\n' +
'[\n' +
' { "type": "QUERY_EXPRESSION", "name": "Service", "expression": "SpanName",\n' +
' "sourceId": "<trace-source-id>", "whereLanguage": "sql",\n' +
' "appliesToSourceIds": ["<trace-source-id>"] }\n' +
']',
);

Expand Down
68 changes: 68 additions & 0 deletions packages/api/src/routers/external-api/__tests__/dashboards.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -847,6 +847,9 @@ describe('External API v2 Dashboards - old format', () => {
expression: 'service_name',
sourceId: traceSource._id.toString(),
sourceMetricType: undefined,
// Scope to a single source (the common case for mixed-source
// dashboards) — exercises the array round-trip.
appliesToSourceIds: [traceSource._id.toString()],
},
{
type: 'QUERY_EXPRESSION' as const,
Expand All @@ -855,6 +858,11 @@ describe('External API v2 Dashboards - old format', () => {
sourceId: traceSource._id.toString(),
where: "environment = 'production'",
whereLanguage: 'sql' as const,
// Scope to multiple sources to exercise multi-entry arrays.
appliesToSourceIds: [
traceSource._id.toString(),
metricSource._id.toString(),
],
},
],
};
Expand Down Expand Up @@ -885,14 +893,25 @@ describe('External API v2 Dashboards - old format', () => {
);
expect(response.body.data.filters[0].name).toBe('Environment');
expect(response.body.data.filters[0].expression).toBe('environment');
// Filter 0 omitted appliesToSourceIds (broadcast-to-all) — must NOT be
// materialized as an empty array on read; the field stays absent so
// the default semantics survive a save/load round-trip.
expect(response.body.data.filters[0].appliesToSourceIds).toBeUndefined();
expect(response.body.data.filters[1].name).toBe('Service Filter');
expect(response.body.data.filters[1].expression).toBe('service_name');
expect(response.body.data.filters[1].appliesToSourceIds).toEqual([
traceSource._id.toString(),
]);
expect(response.body.data.filters[2].name).toBe('Region (Filtered)');
expect(response.body.data.filters[2].expression).toBe('region');
expect(response.body.data.filters[2].where).toBe(
"environment = 'production'",
);
expect(response.body.data.filters[2].whereLanguage).toBe('sql');
expect(response.body.data.filters[2].appliesToSourceIds).toEqual([
traceSource._id.toString(),
metricSource._id.toString(),
]);

const getResponse = await authRequest(
'get',
Expand Down Expand Up @@ -1045,13 +1064,19 @@ describe('External API v2 Dashboards - old format', () => {
name: 'Updated Filter 1',
expression: 'environment',
sourceId: traceSource._id.toString(),
// Broadcast filter: appliesToSourceIds intentionally omitted.
},
{
id: filterId2,
type: 'QUERY_EXPRESSION' as const,
name: 'Updated Filter 2',
expression: 'service_name',
sourceId: traceSource._id.toString(),
// Multi-source scope to exercise array round-trip on PUT.
appliesToSourceIds: [
traceSource._id.toString(),
metricSource._id.toString(),
],
},
],
},
Expand All @@ -1069,13 +1094,20 @@ describe('External API v2 Dashboards - old format', () => {
expression: 'environment',
sourceId: traceSource._id.toString(),
});
// Broadcast filter must stay broadcast on read — the field must not
// be materialized into an empty array by save/load.
expect(response.body.data.filters[0].appliesToSourceIds).toBeUndefined();
expect(response.body.data.filters[1]).toMatchObject({
id: expect.any(String),
type: 'QUERY_EXPRESSION',
name: 'Updated Filter 2',
expression: 'service_name',
sourceId: traceSource._id.toString(),
});
expect(response.body.data.filters[1].appliesToSourceIds).toEqual([
traceSource._id.toString(),
metricSource._id.toString(),
]);

const getResponse = await authRequest(
'get',
Expand All @@ -1094,6 +1126,8 @@ describe('External API v2 Dashboards - old format', () => {
name: 'Existing Filter 1',
expression: 'environment',
sourceId: traceSource._id.toString(),
// Stored with a scope — a no-filters PUT must preserve it intact.
appliesToSourceIds: [traceSource._id.toString()],
},
{
id: existingFilterId2,
Expand Down Expand Up @@ -3039,6 +3073,9 @@ describe('External API v2 Dashboards - new format', () => {
expression: 'service_name',
sourceId: traceSource._id.toString(),
sourceMetricType: undefined,
// Scope to a single source (the common case for mixed-source
// dashboards) — exercises the array round-trip.
appliesToSourceIds: [traceSource._id.toString()],
},
{
type: 'QUERY_EXPRESSION' as const,
Expand All @@ -3047,6 +3084,11 @@ describe('External API v2 Dashboards - new format', () => {
sourceId: traceSource._id.toString(),
where: "environment = 'production'",
whereLanguage: 'sql' as const,
// Scope to multiple sources to exercise multi-entry arrays.
appliesToSourceIds: [
traceSource._id.toString(),
metricSource._id.toString(),
],
},
],
};
Expand Down Expand Up @@ -3077,14 +3119,25 @@ describe('External API v2 Dashboards - new format', () => {
);
expect(response.body.data.filters[0].name).toBe('Environment');
expect(response.body.data.filters[0].expression).toBe('environment');
// Filter 0 omitted appliesToSourceIds (broadcast-to-all) — must NOT be
// materialized as an empty array on read; the field stays absent so
// the default semantics survive a save/load round-trip.
expect(response.body.data.filters[0].appliesToSourceIds).toBeUndefined();
expect(response.body.data.filters[1].name).toBe('Service Filter');
expect(response.body.data.filters[1].expression).toBe('service_name');
expect(response.body.data.filters[1].appliesToSourceIds).toEqual([
traceSource._id.toString(),
]);
expect(response.body.data.filters[2].name).toBe('Region (Filtered)');
expect(response.body.data.filters[2].expression).toBe('region');
expect(response.body.data.filters[2].where).toBe(
"environment = 'production'",
);
expect(response.body.data.filters[2].whereLanguage).toBe('sql');
expect(response.body.data.filters[2].appliesToSourceIds).toEqual([
traceSource._id.toString(),
metricSource._id.toString(),
]);

const getResponse = await authRequest(
'get',
Expand Down Expand Up @@ -3206,13 +3259,19 @@ describe('External API v2 Dashboards - new format', () => {
name: 'Updated Filter 1',
expression: 'environment',
sourceId: traceSource._id.toString(),
// Broadcast filter: appliesToSourceIds intentionally omitted.
},
{
id: filterId2,
type: 'QUERY_EXPRESSION' as const,
name: 'Updated Filter 2',
expression: 'service_name',
sourceId: traceSource._id.toString(),
// Multi-source scope to exercise array round-trip on PUT.
appliesToSourceIds: [
traceSource._id.toString(),
metricSource._id.toString(),
],
},
],
},
Expand All @@ -3230,13 +3289,20 @@ describe('External API v2 Dashboards - new format', () => {
expression: 'environment',
sourceId: traceSource._id.toString(),
});
// Broadcast filter must stay broadcast on read — the field must not
// be materialized into an empty array by save/load.
expect(response.body.data.filters[0].appliesToSourceIds).toBeUndefined();
expect(response.body.data.filters[1]).toMatchObject({
id: expect.any(String),
type: 'QUERY_EXPRESSION',
name: 'Updated Filter 2',
expression: 'service_name',
sourceId: traceSource._id.toString(),
});
expect(response.body.data.filters[1].appliesToSourceIds).toEqual([
traceSource._id.toString(),
metricSource._id.toString(),
]);

const getResponse = await authRequest(
'get',
Expand All @@ -3255,6 +3321,8 @@ describe('External API v2 Dashboards - new format', () => {
name: 'Existing Filter 1',
expression: 'environment',
sourceId: traceSource._id.toString(),
// Stored with a scope — a no-filters PUT must preserve it intact.
appliesToSourceIds: [traceSource._id.toString()],
},
{
id: existingFilterId2,
Expand Down
11 changes: 11 additions & 0 deletions packages/api/src/routers/external-api/v2/dashboards.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1359,6 +1359,17 @@ function getSourceConnectionMismatches(
* description: Language of the where condition
* default: "sql"
* example: "lucene"
* appliesToSourceIds:
* type: array
* items:
* type: string
* description: |
* Optional list of source IDs this filter applies to. Omit or provide
* an empty array to apply the filter to ALL tiles regardless of source.
* A non-empty array restricts the filter to only tiles whose source ID
* is in the list; tiles using other sources are not affected by the
* selected filter value(s).
* example: ["65f5e4a3b9e77c001a111111"]
*
* Filter:
* allOf:
Expand Down
Loading
Loading