fix: dashboard performance

[dpgaspar]

SUMMARY

This PR introduces a substantial performance improvement to the dashboard list page.
We have noticed that json_metadata and position_json can contain huge payloads on dashboards that contain a large amount of charts (hundreds). This can heavily impact performance.

The easier path would be to just remove those columns from list_columns on the backend, but thats a breaking change. Also thumbnail_url uses position_json and json_metadata to compute the MD5 digest. So currently it's not possible to improve the REST API performance itself by default.

it's possible on FAB to just request the columns we want using the columns query parameter, but the SQL selected columns would still use the configures list_columns or list_select_columns. On 4.5.0 I've introduced a new query parameter named select_columns that works exactly like columns but will actually on query the requested columns (without never "escaping" the list_columns set configured on the backend, for security reasons).

Using the example dashboards:

Requesting just the columns we need defined on this PR, avg API response is ~16ms

Using the API by default (backend defined columns), avg API response is ~120ms

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

Attention: Patch coverage is 86.66667% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 70.29%. Comparing base (76d897e) to head (f366583).
Report is 233 commits behind head on master.

Files	Patch %	Lines
...frontend/src/features/dashboards/DashboardCard.tsx	84.61%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #28609      +/-   ##
==========================================
+ Coverage   60.48%   70.29%   +9.81%     
==========================================
  Files        1931     1949      +18     
  Lines       76236    77635    +1399     
  Branches     8568     8738     +170     
==========================================
+ Hits        46114    54577    +8463     
+ Misses      28017    20930    -7087     
- Partials     2105     2128      +23     

Flag	Coverage Δ
hive	49.07% <ø> (-0.10%)	⬇️
javascript	57.87% <86.66%> (+0.16%)	⬆️
mysql	77.16% <ø> (?)
postgres	77.29% <ø> (?)
presto	53.61% <ø> (-0.19%)	⬇️
python	83.53% <ø> (+20.05%)	⬆️
sqlite	76.74% <ø> (?)
unit	59.01% <ø> (+1.38%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[eschutho]

/testenv up

[github-actions]

@eschutho Ephemeral environment spinning up at http://35.95.35.109:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

[mistercrunch]

just noting that this particular pinned dep has been flickering, meaning pip-compile-multi isn't as deterministic as I'd like for it to be, for reasons I don't really understand

[mistercrunch]

LGTM overall. It's nice to be able to specify the columns needed for the list view

[github-actions]

Ephemeral environment shutdown and build artifacts deleted.