perf: Improve dashboard performance by decreasing rerenders

[kgabryje]

SUMMARY

SUPERSEDED BY PRs: #31240, #31241, #31242, #31243, #31244.
This PR is not to be merged - aggregates 5 smaller PRs to be referenced as total performance improvement.

Due to many inefficient redux selectors and other factors causing numerous rerenders, large dashboards were running very slow.

This PR refactors many components related to Dashboard and native filters - some components are rewritten to functional to utilize React's memoization, Redux selectors optimized to select smaller objects that change less frequently, optimized frequently called functions.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

The tests were conducted on a large dashboard - 300+ charts, 40+ filters. Test steps: reload the page, let the 1st tab load, switch to a different tab, let everything load.
Tested on my M2 Pro Macbook - since it's likely that an average Superset user works on a slower computer, I also run the tests with 4x and 20x CPU throttling.
Testing tool - chrome dev tools, performance tab

1. No throttling
   Before - total scripting time 2.73s (screenshot 1), each chart holder blocks the main thread for ~70ms
   After - total scripting time 1.48s (screenshot 2), each chart holder blocks the main thread for ~5ms

2. 4x throttling
   Before - total scripting time 11.93s (screenshot 3), each chart holder blocks the main thread for ~300ms
   After - total scripting time 6.23s (screenshot 4), each chart holder blocks the main thread for ~20ms

3. 20x throttling
   Before - total scripting time 82s (screenshot 5), each chart holder blocks the main thread for ~2s
   After - total scripting time 32s (screenshot 6), each chart holder blocks the main thread for ~100ms

SCREENSHOTS BEFORE:

1. No throttling

2. 4x throttling

3. 20x throttling

SCREENSHOTS AFTER:

1. No throttling

2. 4x throttling

3. 20x throttling

TESTING INSTRUCTIONS

No functional changes - all dashboard functionalities should work like before.

Recommend thorough testing of:

• Filter interactions
• Chart loading/rendering
• Dashboard navigation
• Cross-filter functionality
• Dashboard state persistence

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[villebro]

These PRs from Kamil always make me soo happy! ❤️

[villebro]

/testenv up

[villebro]

I did my best to find anything to comment or even nit about in this PR, but this looks really solid. I will have to resort to testing to see there's anything that looks out of the ordinary. Will approve once I'm done with the testing rounds..

[github-actions]

@villebro Ephemeral environment spinning up at http://52.38.80.189:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

[fisjac]

Nice!

[fisjac]

nice!

[fisjac]

Nice improvement. Nit: should this be named nativeFilters outside of the context of this PR, I don't think "optimized" make sense.

[villebro]

/testenv up

[github-actions]

@villebro Ephemeral environment spinning up at http://35.91.239.242:8080. Credentials are admin/admin. Please allow several minutes for bootstrapping and startup.

[kgabryje]

@villebro unfortunately the ephemeral environments have been broken for some time now 🙁

[kgabryje]

Neat AI summary provided by @rusackas 🙂

Key changes include:

1. Memoization optimizations:
   • Added useMemo/useCallback hooks throughout components
   • Memoized selectors with createSelector
   • Reduced unnecessary re-renders
2. Component modernization:
   • Converted class components to functional components with hooks
   • Removed redundant Redux connections in favor of useSelector/useDispatch
   • Simplified prop passing and state management
3. Architecture improvements:
   • Moved Chart container logic directly into component
   • Simplified data flow and prop drilling
   • Better separation of concerns

The main potential risk areas to watch:

1. FilterBar and native filtering changes - ensure filter state management remains consistent with previous behavior
2. Chart component refactoring - verify chart rendering and updates work correctly, especially around:
   • Data loading states
   • Filter interactions
   • Resize handling
3. Tab/Dashboard navigation - confirm tab state management and routing remains intact

Recommend thorough testing of:
- Filter interactions
- Chart loading/rendering
- Dashboard navigation
- Cross-filter functionality
- Dashboard state persistence