Skip to content

fix: support nested function calls in cache_key_wrapper#38569

Merged
michael-s-molina merged 4 commits intoapache:masterfrom
villebro:villebro/cache-key-wrapper-nested-fn
Mar 12, 2026
Merged

fix: support nested function calls in cache_key_wrapper#38569
michael-s-molina merged 4 commits intoapache:masterfrom
villebro:villebro/cache-key-wrapper-nested-fn

Conversation

@villebro
Copy link
Member

@villebro villebro commented Mar 11, 2026

User description

SUMMARY

#30549 introduced a regression that caused nested function calls inside cache_key_wrapper() to go undetected. This means, that while the following Jinja snippets were functionally identical, only the former was flagged as containing extra cache keys:

{% set abc = myfunc() %}
{{ cache_key_wrapper(abc) }}

is the same as

{{ cache_key_wrapper(myfunc()) }}

The change relaxes the regex to detect both cases correctly, and adds tests to catch a similar regression going forward.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue: closes Superset no longer accepts custom jinja template functions for caching #33775
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

CodeAnt-AI Description

Detect nested templated function calls when computing extra cache keys

What Changed

  • Regex that finds templated helper calls (cache_key_wrapper, url_param, current_user_*) was relaxed so calls wrapped inside other calls are detected (e.g., cache_key_wrapper(myfunc()) and url_param(get_param('x'))).
  • Added parametrized unit tests that verify detection of these nested calls and avoid false positives for plain strings or non-templated expressions.

Impact

✅ Fewer incorrect cache hits
✅ More accurate cache key detection for templated queries
✅ Fewer missed extra cache keys from nested function calls

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

@codeant-ai-for-open-source codeant-ai-for-open-source bot added the size:M This PR changes 30-99 lines, ignoring generated files label Mar 11, 2026
@dosubot dosubot bot added the global:jinja Related to Jinja templating label Mar 11, 2026
@codeant-ai-for-open-source
Copy link
Contributor

Sequence Diagram

This PR updates the ExtraCache regex so templated helper calls, including nested function calls, are detected and incorporated into query cache keys, reducing incorrect cache hits. The diagram shows how query execution now uses the updated detection when building cache keys.

sequenceDiagram
    participant Client
    participant API
    participant TemplateProcessor
    participant ExtraCache
    participant Cache

    Client->>API: Run templated query
    API->>TemplateProcessor: Render template and prepare cache key
    TemplateProcessor->>ExtraCache: Scan template for cache helpers
    ExtraCache->>ExtraCache: Match helper calls including nested functions
    ExtraCache-->>TemplateProcessor: Indicate extra cache keys present
    TemplateProcessor->>Cache: Build cache key with extra values
    Cache-->>API: Return cached or fresh result
    API-->>Client: Return query result
Loading

Generated by CodeAnt AI

@codecov
Copy link

codecov bot commented Mar 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.42%. Comparing base (9215eb5) to head (c0e0f97).
⚠️ Report is 8 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #38569      +/-   ##
==========================================
- Coverage   65.01%   64.42%   -0.60%     
==========================================
  Files        1817     2529     +712     
  Lines       72317   128889   +56572     
  Branches    23031    29707    +6676     
==========================================
+ Hits        47020    83032   +36012     
- Misses      25297    44413   +19116     
- Partials        0     1444    +1444     
Flag Coverage Δ
hive 40.77% <ø> (?)
mysql 61.93% <ø> (?)
postgres 62.00% <ø> (?)
presto 40.79% <ø> (?)
python 63.63% <ø> (?)
sqlite 61.63% <ø> (?)
unit 100.00% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@bito-code-review bito-code-review bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #2600f9

Actionable Suggestions - 2
  • tests/unit_tests/jinja_context_test.py - 1
  • tests/integration_tests/sqla_models_tests.py - 1
Review Details
  • Files reviewed - 3 · Commit Range: c071f55..c071f55
    • superset/jinja_context.py
    • tests/integration_tests/sqla_models_tests.py
    • tests/unit_tests/jinja_context_test.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@bito-code-review
Copy link
Contributor

bito-code-review bot commented Mar 11, 2026

Code Review Agent Run #aa597b

Actionable Suggestions - 0
Review Details
  • Files reviewed - 2 · Commit Range: c071f55..5e41c2f
    • tests/unit_tests/jinja_context_test.py
    • tests/integration_tests/sqla_models_tests.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@villebro
Copy link
Member Author

@neuromantik33 please review if you have the time

@codeant-ai-for-open-source codeant-ai-for-open-source bot added size:S This PR changes 10-29 lines, ignoring generated files and removed size:M This PR changes 30-99 lines, ignoring generated files labels Mar 11, 2026
@codeant-ai-for-open-source
Copy link
Contributor

Sequence Diagram

This PR updates the ExtraCache regex so that nested Jinja helper calls (such as cache_key_wrapper wrapping another function) are detected and included when computing query cache keys, improving cache correctness for templated SQL.

sequenceDiagram
    participant User
    participant TemplateProcessor
    participant ExtraCache
    participant Cache

    User->>TemplateProcessor: Submit SQL with Jinja template
    TemplateProcessor->>TemplateProcessor: Scan template with ExtraCache regex
    alt helper calls found (including nested)
        TemplateProcessor->>ExtraCache: Render helper calls and capture values
        ExtraCache->>Cache: Append values to extra cache keys
    end
    TemplateProcessor-->>User: Return result using computed cache key
Loading

Generated by CodeAnt AI

@bito-code-review
Copy link
Contributor

bito-code-review bot commented Mar 11, 2026

Code Review Agent Run #acb8a5

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: 5e41c2f..c0e0f97
    • tests/unit_tests/jinja_context_test.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@michael-s-molina michael-s-molina merged commit a9def2f into apache:master Mar 12, 2026
74 of 75 checks passed
@villebro villebro deleted the villebro/cache-key-wrapper-nested-fn branch March 13, 2026 16:27
michael-s-molina pushed a commit that referenced this pull request Mar 17, 2026
@michael-s-molina michael-s-molina moved this from Done to Cherried in Superset 6.1.0 Release Bugs Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

global:jinja Related to Jinja templating size/M size:S This PR changes 10-29 lines, ignoring generated files

Projects

Development

Successfully merging this pull request may close these issues.

Superset no longer accepts custom jinja template functions for caching

3 participants