[FEATURE] Initial ScatterChart Panel Plugin #1672

zhuje · 2023-12-21T05:54:02Z

Description

TLDR: Initial ScatterChart Panel Plugin

Add Panel Plugin for Scatter Chart
- Scatter Chart only handles Trace data for now
- Scatter Chart currently doesn't offer any spec options to format chart (e.g., y-axis label, min value, max value)
Add Tempo Query Editor for query editing in UI
- There is no auto-complete extension in CodeMirror for Tempo

Note

ScatterChart Panel Plugin is based on the work done by Steve Cobb from [perses-embed-example](https://github.com/sjcobb/perses-embed-example).

Backend Changes

Add schemas for scatter plot panels in schemas/panels/scatterplot . In this initial iteration, no spec options are added to this panel.

Frontend Changes

1. Add scatter plot panel for data visualization

Files Changes in perses/ui/panels-plugin/src/scatterplot
- ScatterChart.ts — define a PanelPlugin Chart. The PluginRegistry uses this file to fetch the component dynamically (see the file PanelContent.tsx and the usePlugin() function).
- ScatterChartPanel.tsx — define a PanelComponent, transforms the Tempo API data into a ‘dataset’ that can be consumed by Apache ECharts. Pass dataset and additional visualization options as props to component.
  - DataSet
    - Documentation for data structures accepted by Apache ECharts: https://echarts.apache.org/handbook/en/concepts/dataset
  - Formatting
    - Datapoint size (bubble size). A single data point is a trace. The number of spans in a trace dictates the data point size.
    - Data point color. If a trace contains an error, it will color the datapoint in red.
    - Examples for chart formatting in Apache ECharts: https://echarts.apache.org/examples/en/index.html#chart-type-scatter
- ScatterChartPanel.test.tsx — test if ScatterChartPanel can render correctly
- Scatterplot.tsx - accept props from Scatterplot and format the Apache EChart with the props.
- scatter-chart-model.ts - define models for components in ScatterChart.ts

1B. Component TreeView

ScatterChart
- ScatterChartPanel
  - ScatterPlot

2. Changes to add a UI panel editing in a scatter plot with Tempo Data
Note: These changes were based on Time Series Query Editor. In the future, if additional data types are added, we should consider how to create more reusable components (i.e., if we can make a general query editor that is less tightly coupled to a specific query type).

File Changes in perses/ui/plugin-system/src/components
- PanelSpecEditor.tsx — modified to include a switch statement to render the correct query editor based on the dashboard definition spec > query kind (e.g., TimeSeriesQuery, TraceQuery). For example : switch(queryType) >> case ‘TraceQuery’ >> render <TraceQueryEditor>
- TraceQueryEditor.tsx — holds the logic for handling add/removing queries in trace query editor and passes these as props to TraceQueryInput
- TraceQueryInput.tsx— Renders collapsing and delete buttons. Contains the PluginEditor component, which uses the Panel’s query type (i.e., TraceQuery) to render the corresponding TraceQuery editing components
  - PluginEditor.tsx
    - PluginKindSelect.tsx — renders a drop-down containing kinds for our query type. For example, <PluginKind queryType={TraceQuery}/> drop-down will render TempoTraceQuery.
    - PluginSpecEditor.tsx — dynamically renders the corresponding OptionsEditorComponent given a query type. For example, <PluginSpecEditor queryType={TraceQuery}/> will fetch the query plugin from the file TempoTraceQuery.tsx and get the OptionsEditorComponent: TempoTraceQueryEditor
File Changes in perses/ui/tempo-plugin/src/tempo-trace-query
- TempoTraceQuery.tsx — define OptionsEditorComponent: TempoTraceQueryEditor in the query plugin
- TempoTraceQueryEditor.tsx — handles data source change, contains
- DashboardTraceQueryEditor.tsx — contains and
- TraceQLEditor.tsx— embed , which is a text input for a query. Currently, there is no autocomplete extension for Tempo.

3. Enable TimeRangeSelector on TraceQuery

File changes for /perses/ui/plugin-system/src/runtime
- trace-queries.tsx - fetch selected time range (e.g. ‘last 1 hour, last 1 day’) and pass it as a prop to the trace fetching function getTraceData() in the get-trace-data.tsx file
File changes for perses/ui/tempo-plugin/src/tempo-trace-query
- get-trace-data.tsx — contains getTraceData(). Modify the query to include a ‘start’ and ‘end’ time. For example {fooQuery}&start={unixStartTime}&end{unixEndTime}. Pass this query to client.getEnrichedTraceQuery(getQuery(), datasourceUrl) to fetch trace data from API.

Screenshots

Figure 1. Scatter Chart
Datapoint size corresponds to the number of spans within a trace.
Red datapoints indicate the trace contains and error.

Figure 2. Trace Query Editor

Checklist

Pull request has a descriptive title and context useful to a reviewer.
Pull request title follows the [<catalog_entry>] <commit message> naming convention using one of the
following catalog_entry values: FEATURE, ENHANCEMENT, BUGFIX, BREAKINGCHANGE, DOC,IGNORE.
All commits have [DCO signoffs](https://github.com/probot/dco#how-it-works).

UI Changes

Changes that impact the UI include screenshots and/or screencasts of the relevant changes.
Code follows the [UI guidelines](https://github.com/perses/perses/blob/main/ui/ui-guidelines.md).
Visual tests are stable and unlikely to be flaky.
See [Storybook](https://github.com/perses/perses/tree/main/ui/storybook#visual-tests)
and [e2e](https://github.com/perses/perses/tree/main/ui/e2e#visual-tests) docs for more details. Common issues
include:
- Is the data inconsistent? You need to mock API requests.
- Does the time change? You need to use consistent time values or mock time utilities.
- Does it have loading states? You need to wait for loading to complete.

-- Add Panel Plugin for Scatter Chart -- Add Tempo Query Editor for UI Editing Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

ui/panels-plugin/src/plugins/scatterplot/ScatterChartPanel.tsx

docs/plugin/panel.md

ui/plugin-system/src/components/TraceQueryEditor/TraceQueryEditor.tsx

ui/tempo-plugin/src/plugins/tempo-trace-query/get-trace-data.ts

Merge from main. Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

sjcobb · 2024-01-12T01:38:19Z

ui/panels-plugin/src/plugins/scatterplot/ScatterChartPanel.tsx

+  startTime: Date;
+}
+
+export function getUnixTimeRange(timeRange: AbsoluteTimeRange) {


Can't remember exactly, but we might have a similar util in @perses-dev/core

I’ve removed this in the latest commit. This was a helper function used by greaterThanSevenDays().

sjcobb · 2024-01-12T01:44:33Z

ui/panels-plugin/src/plugins/scatterplot/ScatterChartPanel.tsx

+  // 1. Map x,y coordinates
+  // 2. Datapoint size corresponds to the number of spans in a trace
+  // 3. Color datapoint red if the trace contains an error
+  const seriesTemplate: SeriesOption = {


The dataset lines above are in a useMemo so that's good, but these other lines could benefit from being memozied too. It would break the rules of hooks though so having a separate component like ScatterChartPanelLoaded.tsx would be the way to go (other panel types would benefit from that too)

TBH, I’m a novice with hooks and don’t see how it would break the rules. Could you elaborate on the rules that are being broken and the implementation you’d like to see for the separate component ScatterChartPanelLoaded? In the latest push, I attempted to use the useMemo() hook directly in scatterChartPanel to memoize the series formatting, and it didn’t throw any errors…

I am not sure how the rules are being broken because the memoized function is not inside a condition, loop, or event handler, and there are no other hooks within the function. And I moved the error checks that would return the function early, after the hooks.

Oh great! I thought it was after the loading if conditions but I might have misread too, if no errors were thrown or anything then you should be good

sjcobb · 2024-01-12T01:47:27Z

ui/panels-plugin/src/plugins/scatterplot/ScatterChartPanel.tsx

+          return 'red';
+        }
+        // Else return default color
+        const defaultColor = '#56B4E9';


chartsTheme is nice to leverage for these types of colors since it allows embed users to customize their palette, fonts, etc. (ex of defaultColor for thresholds)

Absolutely, agree. Great catch; this is updated in the latest commit.

Awesome and you can always add new colors too if what’s there isn’t enough

sjcobb · 2024-01-12T01:49:17Z

ui/panels-plugin/src/plugins/scatterplot/Scatterplot.tsx

+    dataset: options.dataset,
+    series: options.series,
+    grid: {
+      bottom: 40,


Similar comment about possibly leveraging chartsTheme if possible

Edit: NVM, ignore me, I see that grid is pretty custom here due to dataZoom

sjcobb · 2024-01-12T01:57:14Z

ui/tempo-plugin/src/plugins/tempo-trace-query/query-editor-model.ts

+
+  // Propagate changes to the query's value when the input is blurred to avoid constantly re-running queries in the
+  // PanelPreview
+  const handleQueryBlur = () => {


Side note: sometimes this onBlur logic annoys me in our Prometheus editor and I just want an explicit 'Run queries' button 😂

I agree. This could be generated as a separate issue. Updating the panel to have an explicit 'Run queries' button might also be more intuitive for the user.

sjcobb

🚀

Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

sjcobb

LGTM!

Nexucis

incredible work @zhuje thank you so much for taking the time to do it !

And thank you @sjcobb for reviewing and @zhuje for your patience

Signed-off-by: Steven Cobb <sjcobb.dev@gmail.com>

* [BUGFIX] Tooltip positioning broken in Firefox Signed-off-by: Steven Cobb <sjcobb.dev@gmail.com> * fix lint from Scatterplot #1672, remove totalSeries Signed-off-by: Steven Cobb <sjcobb.dev@gmail.com> --------- Signed-off-by: Steven Cobb <sjcobb.dev@gmail.com>

[FEATURE] Initial ScatterChart Panel Plugin

b7397d6

-- Add Panel Plugin for Scatter Chart -- Add Tempo Query Editor for UI Editing Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

sjcobb self-requested a review December 23, 2023 03:43

sjcobb reviewed Dec 23, 2023

View reviewed changes

ui/panels-plugin/src/plugins/scatterplot/ScatterChartPanel.tsx Outdated Show resolved Hide resolved

sjcobb reviewed Dec 23, 2023

View reviewed changes

docs/plugin/panel.md Show resolved Hide resolved

sjcobb reviewed Dec 23, 2023

View reviewed changes

ui/plugin-system/src/components/TraceQueryEditor/TraceQueryEditor.tsx Show resolved Hide resolved

sjcobb reviewed Dec 23, 2023

View reviewed changes

ui/tempo-plugin/src/plugins/tempo-trace-query/get-trace-data.ts Show resolved Hide resolved

zhuje added 2 commits January 9, 2024 09:02

Merge branch 'main' into scatterplot_pr

17a2cdc

Update package-lock.json from main.

6e3628d

Merge from main. Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

zhuje force-pushed the scatterplot_pr branch from 2657ee6 to 6e3628d Compare January 9, 2024 18:48

zhuje closed this Jan 10, 2024

zhuje reopened this Jan 10, 2024

Remove function that prevents time ranges selections beyond 7 days.

686701e

Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

zhuje force-pushed the scatterplot_pr branch from fbb9150 to 686701e Compare January 11, 2024 16:29

zhuje requested a review from sjcobb January 11, 2024 16:43

sjcobb reviewed Jan 12, 2024

View reviewed changes

sjcobb approved these changes Jan 12, 2024

View reviewed changes

Update ScatterChartPanel by memoizing series formatting.

399d007

Signed-off-by: Jenny Zhu <jenny.a.zhu@gmail.com>

zhuje requested a review from sjcobb January 13, 2024 18:52

sjcobb approved these changes Jan 14, 2024

View reviewed changes

Nexucis approved these changes Jan 17, 2024

View reviewed changes

Nexucis added this pull request to the merge queue Jan 17, 2024

Merged via the queue into perses:main with commit aba5521 Jan 17, 2024
18 checks passed

sjcobb added a commit that referenced this pull request Feb 8, 2024

fix lint from Scatterplot #1672, remove totalSeries

e0b2075

Signed-off-by: Steven Cobb <sjcobb.dev@gmail.com>

sjcobb mentioned this pull request Feb 8, 2024

[BUGFIX] Tooltip positioning broken in Firefox #1750

Merged

6 tasks

sjcobb mentioned this pull request Jun 16, 2024

[FEATURE] TraceTable panel #2061

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] Initial ScatterChart Panel Plugin #1672

[FEATURE] Initial ScatterChart Panel Plugin #1672

zhuje commented Dec 21, 2023

sjcobb Jan 12, 2024

zhuje Jan 13, 2024

sjcobb Jan 12, 2024

zhuje Jan 13, 2024 •

edited

Loading

sjcobb Jan 13, 2024

sjcobb Jan 12, 2024

zhuje Jan 13, 2024

sjcobb Jan 13, 2024

sjcobb Jan 12, 2024 •

edited

Loading

sjcobb Jan 12, 2024

zhuje Jan 13, 2024

sjcobb left a comment

sjcobb left a comment

Nexucis left a comment

[FEATURE] Initial ScatterChart Panel Plugin #1672

[FEATURE] Initial ScatterChart Panel Plugin #1672

Conversation

zhuje commented Dec 21, 2023

Description

TLDR: Initial ScatterChart Panel Plugin

Note

Backend Changes

Frontend Changes

Screenshots

Checklist

UI Changes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhuje Jan 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjcobb Jan 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjcobb left a comment

Choose a reason for hiding this comment

sjcobb left a comment

Choose a reason for hiding this comment

Nexucis left a comment

Choose a reason for hiding this comment

zhuje Jan 13, 2024 •

edited

Loading

sjcobb Jan 12, 2024 •

edited

Loading