Improve time scale ticks behavior #38911

alxnddr · 2024-02-19T15:23:49Z

Closes

[dc.js migration] incorrect formatting on x-axis ticks using native query date columns #38843
[dc.js migration] time series charts with week bucketing show shifted x-axis ticks #38841

Description

The default ECharts tick selection behavior was confusing on weekly and quarterly data because the ticks did not match the actual data:

This PR fixes these cases and overall improves time scale ticks behavior.

Handle weeks and quarters

For these units we force ECharts to render ticks with smaller granularity and then in the formatter function we select desired ticks only.
Weeks: we select daily ticks that have the same day of week as the underlying data.
Quarters: we select monthly ticks that are start of quarters.

Handle long durations

To render long durations relatively to the unit we use the largest possible unit for ticks that exists 3 times within the data which guarantees at least two ticks on the chart.
For example, given the dataset has weekly data but the range of the dataset is 4 years. If we render weekly ticks, they will have some random values that will match start of some weeks of the data. It will be harder to comprehend than just evenly spaced yearly ticks.

The example below demonstrates why it is needed. It has weekly data over a few years.

Without switching to a larger unit for ticks

After switching to a larger unit for ticks

github-actions · 2024-02-19T15:35:47Z

🚨 Visual Regression Tests failed. Get the report artifact

github-actions · 2024-02-19T22:28:25Z

🚨 Visual Regression Tests failed. Get the report artifact

github-actions · 2024-02-20T04:55:32Z

🚨 Visual Regression Tests failed. Get the report artifact

github-actions · 2024-02-21T01:18:13Z

🚨 Visual Regression Tests failed. Get the report artifact

github-actions · 2024-02-22T21:36:55Z

🚨 Visual Regression Tests failed. Get the report artifact

github-actions · 2024-02-22T21:57:29Z

Snapshots have been updated

alxnddr · 2024-02-22T22:09:20Z

frontend/src/metabase/visualizations/lib/timeseries.js

-  { interval: "year", count: 50, testFn: d => d.year() % 50 }, // (20) 50 year
-  { interval: "year", count: 100, testFn: d => d.year() % 100 }, // (21) 100 year
+export const TIMESERIES_INTERVALS = [
+  { unit: "ms", count: 1, testFn: d => 0 }, //  (0) millisecond


I removed a few old dc.js files with dead code since they relied on this objects with the old name

Let's move this to a .ts file in the echarts directory

github-actions · 2024-02-22T22:24:38Z

Snapshots have been updated

github-actions · 2024-02-22T22:24:47Z

🚨 Visual Regression Tests failed. Get the report artifact

replay-io · 2024-02-22T22:29:31Z

Status	Complete ↗︎
Commit	`5274b3a`
Results	❌ 180 Failed ⚠️ 1 Flaky ✅ 2036 Passed

EmmadUsmani

Timeseries waterfall charts are no longer working in app (but they do in static viz), for example http://localhost:3000/question/133-waterfall-timeseries-uk-car-accidents

EmmadUsmani · 2024-02-23T17:00:43Z

Also, when we bucket by "week of year" it displays both the first and the last week as "1st"

We can tackle this in another PR if it's not related, just make sure to file an issue

alxnddr · 2024-02-26T13:57:43Z

@EmmadUsmani can you please share more details about the waterfall question that is not working? I don't use this database due to huge delays which makes it unusable for me locally

github-actions · 2024-02-27T19:21:07Z

🚨 Visual Regression Tests failed. Get the report artifact

github-actions · 2024-02-27T20:20:00Z

Snapshots have been updated

EmmadUsmani · 2024-02-28T17:22:42Z

In a lot of cases just the first tick is missing, which looks wrong from a user perspective. For example,

Also, I'm not really sure what's going on in this case. We went from displaying Feb 16th twice to Feb 27th twice. Is Feb 27th the correct date?

EmmadUsmani

Let's resolve the above issue regarding the first date, I think it's a big regression.

alxnddr · 2024-02-28T17:56:30Z

@EmmadUsmani there is no actual regression in storybook or the app but Loki for some reason flakes and drop them, I'll investigate:

Also, I'm not really sure what's going on in this case. We went from displaying Feb 16th twice to Feb 27th twice. Is Feb 27th the correct date?

I think this spec has relative date time x-axis but due to formatting issues it includes current date which is wrong. I'll check & fix, maybe in a separate PR since this is related to actual time scale x-axis

EmmadUsmani · 2024-02-28T18:06:28Z

frontend/src/metabase/visualizations/lib/timeseries.js

-  { interval: "year", count: 50, testFn: d => d.year() % 50 }, // (20) 50 year
-  { interval: "year", count: 100, testFn: d => d.year() % 100 }, // (21) 100 year
+export const TIMESERIES_INTERVALS = [
+  { unit: "ms", count: 1, testFn: d => 0 }, //  (0) millisecond


Let's move this to a .ts file in the echarts directory

frontend/src/metabase/visualizations/echarts/cartesian/model/axis.ts

frontend/src/metabase/visualizations/echarts/cartesian/option/series.ts

frontend/src/metabase/visualizations/echarts/cartesian/waterfall/model/axis.ts

EmmadUsmani · 2024-02-28T18:48:34Z

I'm seeing this bug on waterfall charts with the total disabled. It only happens in this case, not on all waterfall charts without totals.

alxnddr · 2024-02-29T18:57:41Z

@EmmadUsmani the issue with missing ticks is caused by timezones. Since this PR has grown a lot, I moved the timezone issue in a separate task #39322

alxnddr · 2024-02-29T19:04:23Z

I'm seeing this bug on waterfall charts with the total disabled. It only happens in this case, not on all waterfall charts without totals.

Fixed this, thanks

* handle time series ticks in ECharts * fix waterfall total tick * cleanup * specs * Update Loki Snapshots * review * compensate ticks timezone * update snapshots * spec --------- Co-authored-by: Metabase bot <metabase-bot@metabase.com>

alxnddr self-assigned this Feb 19, 2024

metabase-bot bot added the .Team/DashViz Dashboard and Viz team label Feb 19, 2024

alxnddr force-pushed the force-echarts-show-ticks-only-for-existing-in-dataset-x-values branch from 8eab0b4 to 7b8b778 Compare February 20, 2024 04:46

alxnddr force-pushed the force-echarts-show-ticks-only-for-existing-in-dataset-x-values branch 2 times, most recently from 7310730 to 1d57fb1 Compare February 22, 2024 21:18

alxnddr changed the title ~~force echarts to show x-axis ticks that exist in data~~ Improve time scale ticks behavior Feb 22, 2024

alxnddr added loki-update and removed loki-update labels Feb 22, 2024

alxnddr commented Feb 22, 2024

View reviewed changes

alxnddr requested review from EmmadUsmani, kdoh and cdeweyx February 22, 2024 22:09

alxnddr marked this pull request as ready for review February 22, 2024 22:10

metabase-bot bot added the visual Run Percy visual testing label Feb 22, 2024

alxnddr added loki-update and removed loki-update labels Feb 22, 2024

alxnddr mentioned this pull request Feb 22, 2024

simulate time scale with category scale #38891

Closed

5 tasks

cdeweyx approved these changes Feb 23, 2024

View reviewed changes

EmmadUsmani requested changes Feb 23, 2024

View reviewed changes

alxnddr added the loki-update label Feb 27, 2024

EmmadUsmani requested changes Feb 28, 2024

View reviewed changes

EmmadUsmani reviewed Feb 28, 2024

View reviewed changes

alxnddr force-pushed the force-echarts-show-ticks-only-for-existing-in-dataset-x-values branch from 30ba05b to 12b1e51 Compare February 29, 2024 18:13

alxnddr and others added 8 commits February 29, 2024 15:26

handle time series ticks in ECharts

9967afd

fix waterfall total tick

302843c

cleanup

a1d130b

specs

cd16aa5

Update Loki Snapshots

bb44efb

review

8436c54

compensate ticks timezone

1fd2b54

update snapshots

5f5c06f

alxnddr force-pushed the force-echarts-show-ticks-only-for-existing-in-dataset-x-values branch from 12b1e51 to 5f5c06f Compare February 29, 2024 18:36

spec

5274b3a

alxnddr requested a review from EmmadUsmani February 29, 2024 18:57

EmmadUsmani approved these changes Feb 29, 2024

View reviewed changes

alxnddr merged commit 4e4c662 into echarts Feb 29, 2024
73 of 107 checks passed

alxnddr deleted the force-echarts-show-ticks-only-for-existing-in-dataset-x-values branch February 29, 2024 20:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve time scale ticks behavior #38911

Improve time scale ticks behavior #38911

alxnddr commented Feb 19, 2024 •

edited

github-actions bot commented Feb 19, 2024

github-actions bot commented Feb 19, 2024

github-actions bot commented Feb 20, 2024

github-actions bot commented Feb 21, 2024

github-actions bot commented Feb 22, 2024

github-actions bot commented Feb 22, 2024

alxnddr Feb 22, 2024

EmmadUsmani Feb 28, 2024

github-actions bot commented Feb 22, 2024

github-actions bot commented Feb 22, 2024

replay-io bot commented Feb 22, 2024 •

edited

EmmadUsmani left a comment

EmmadUsmani commented Feb 23, 2024

alxnddr commented Feb 26, 2024

github-actions bot commented Feb 27, 2024

github-actions bot commented Feb 27, 2024

EmmadUsmani commented Feb 28, 2024 •

edited

EmmadUsmani left a comment •

edited

alxnddr commented Feb 28, 2024

EmmadUsmani Feb 28, 2024

EmmadUsmani commented Feb 28, 2024 •

edited

alxnddr commented Feb 29, 2024

alxnddr commented Feb 29, 2024

Improve time scale ticks behavior #38911

Improve time scale ticks behavior #38911

Conversation

alxnddr commented Feb 19, 2024 • edited

Description

Handle weeks and quarters

Handle long durations

github-actions bot commented Feb 19, 2024

github-actions bot commented Feb 19, 2024

github-actions bot commented Feb 20, 2024

github-actions bot commented Feb 21, 2024

github-actions bot commented Feb 22, 2024

github-actions bot commented Feb 22, 2024

alxnddr Feb 22, 2024

Choose a reason for hiding this comment

EmmadUsmani Feb 28, 2024

Choose a reason for hiding this comment

github-actions bot commented Feb 22, 2024

github-actions bot commented Feb 22, 2024

replay-io bot commented Feb 22, 2024 • edited

EmmadUsmani left a comment

Choose a reason for hiding this comment

EmmadUsmani commented Feb 23, 2024

alxnddr commented Feb 26, 2024

github-actions bot commented Feb 27, 2024

github-actions bot commented Feb 27, 2024

EmmadUsmani commented Feb 28, 2024 • edited

EmmadUsmani left a comment • edited

Choose a reason for hiding this comment

alxnddr commented Feb 28, 2024

EmmadUsmani Feb 28, 2024

Choose a reason for hiding this comment

EmmadUsmani commented Feb 28, 2024 • edited

alxnddr commented Feb 29, 2024

alxnddr commented Feb 29, 2024

alxnddr commented Feb 19, 2024 •

edited

replay-io bot commented Feb 22, 2024 •

edited

EmmadUsmani commented Feb 28, 2024 •

edited

EmmadUsmani left a comment •

edited

EmmadUsmani commented Feb 28, 2024 •

edited