[Bug] /api/v1/chart/warm_up_cache generates different cache keys than runtime - Filter Order Mismatch

[curiousnihilist]

Summary

The /api/v1/chart/warm_up_cache endpoint fails to effectively warm cache for charts on dashboards with native filters. Even when providing extra_filters parameter, the generated cache keys differ from those produced at runtime by /api/v1/chart/data.

Environment

• Superset Version: 6.x (master branch)
• Deployment: Local development

Root Cause Analysis

I've identified two root causes for the cache key mismatch:

1. Native Filters Not Extracted

get_dashboard_extra_filters() in superset/views/utils.py:332 only reads legacy default_filters, not native_filter_configuration:

# Current code only handles legacy filters
default_filters = dashboard.params_dict.get("default_filters")  # Legacy only!

# Native filters are stored differently:
# json_metadata.native_filter_configuration[].defaultDataMask.filterState.value

Impact: Charts with native filter defaults get different filter values during warmup vs runtime.

2. Filter Order Mismatch

Even when extra_filters are provided, they're appended to the filter list via .extend(), but browsers send native filters first:

Warmup code (line 101 in warm_up_cache.py):

query.filter.extend(dashboard_filters)  # Appends at END

Result:

Warmup filters:  [filter_a, filter_b, native_filter]  ← extra_filter at END
Browser filters: [native_filter, filter_a, filter_b]  ← native filter FIRST

Since QueryObject.cache_key() hashes the filter list as-is, different order → different MD5 hash → cache miss.

Reproduction Steps

1. Create dashboard with native filter having a default value
2. Add charts to dashboard
3. Call warmup API with extra_filters matching native filter defaults
4. Open dashboard in browser
5. Check cache - always MISS despite warmup (cache keys differ due to filter order)

How to Verify

1. Call warmup API with extra_filters
2. Open dashboard in browser with DevTools Network tab
3. Compare cache_key in /api/v1/chart/data response vs warmup
4. They will differ because warmup appends filters at end, browser sends them first

Verified Fix

I've tested and confirmed the fix works:

The Fix (one line change in superset/commands/chart/warm_up_cache.py line 101):

# Before (appends at end - wrong order)
query.filter.extend(dashboard_filters)

# After (prepends at beginning - matches browser order)
query.filter = list(dashboard_filters) + query.filter

Result: After applying this fix, both warmup and runtime endpoints generate identical cache keys, and is_cached: True is returned confirming cache hit.

Proposed Solutions

Option A: Prepend filters instead of append (Recommended)

In superset/commands/chart/warm_up_cache.py line 101, change:

# Current (appends at end)
query.filter.extend(dashboard_filters)

# Fix (prepends at beginning to match browser order)
query.filter = list(dashboard_filters) + query.filter

Option B: Sort filters before hashing

In QueryObject.cache_key(), sort filters by column name before hashing to make order irrelevant.

Workaround

Currently, the only reliable warmup method is:

1. Use Playwright/Puppeteer to capture exact browser payloads
2. Replay those payloads directly to /api/v1/chart/data

This is not ideal for production environments.

Related Issues

• Cache warm_up logic? #34837 - Cache warm_up logic
• How to Implement Warm-Up Caching in Apache Superset Using Redis and Celery for Dashboards with multiple JINJA params ? #29307 - How to implement warm-up caching

Questions

1. Is there an existing fix or PR addressing this?
2. Would the team accept a PR for Option A (prepend fix)?

---

Labels to add: bug, caching, api

[dosubot[bot]]

Hi @curiousnihilist! I'm Dosu and I'm helping the Apache Superset team.

Your analysis is solid. I can confirm the two root causes you identified:

1. Native filters not handled: get_dashboard_extra_filters() in superset/views/utils.py indeed only reads legacy default_filters and does not process native_filter_configuration [1]. Native filter defaults are completely absent from the warmup path.

2. Filter order affects cache keys: QueryObject.cache_key() hashes the filter list as-is without sorting [2], so any difference in filter ordering between warmup and runtime produces a different cache key.

A note on the specific code location: the current warm_up_cache.py file structure may differ from what you're seeing — the non-legacy path (lines 77-95) doesn't appear to inject dashboard filters at all, rather than appending them in the wrong order [3]. You may be working with a slightly different version or local modifications. Worth double-checking the exact line you're patching.

Regarding your questions:

1. No existing PR addresses this specific filter order / native filter warmup issue. However, there's precedent for cache-key consistency fixes — PR #40993 fixed a similar cache key mismatch caused by mutating orderby expressions during query construction, and PR #41531 added NativeFilterOptionsStrategy to pre-populate native filter option queries (though it didn't change cache-key semantics).

2. Between your two options, Option B (sort filters before hashing) would be the more robust long-term fix since it makes cache keys order-invariant and prevents similar issues from surfacing elsewhere. Option A (prepend) is more targeted and lower-risk but couples the warmup code to browser behavior. Both are reasonable — the team would likely have opinions on which approach is preferred in a PR review.

Also worth noting: issue #34837 highlights a broader challenge — cache warmup runs under a system user context, so if Row-Level Security (RLS) is enabled, cache keys will still differ per user regardless of filter order fixes. Something to keep in mind if RLS is part of your setup.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[rusackas]

Confirmed this is real, verified both root causes myself against current master rather than just trusting the write-up (and the write-up holds up well, though I'd push back a little on the dosu bot's aside — it says the non-legacy warmup path "doesn't appear to inject dashboard filters at all," but that's wrong, it still does the .extend() append). This should've been an issue, not a discussion, so I filed one: #43024.

Also found something useful for whoever picks up the fix: there's already a correct native-filter extractor wired into the real /api/v1/chart/data endpoint (get_dashboard_filter_context/apply_dashboard_filter_context in superset/charts/data/dashboard_filter_context.py). ChartWarmUpCacheCommand just isn't using it — reusing that instead of patching get_dashboard_extra_filters directly is probably the more correct fix than the one-line prepend you proposed.

Opened a test-only PR pinning the native-filter part of this: #43025. Left the filter-order part (your Option A/B) as a follow-up once that's fixed, since both need to land for warming to actually work.