Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use pushdown params to disable time range constraint pushdown #1216

Merged
merged 5 commits into from
May 24, 2024
Merged

Use pushdown params to disable time range constraint pushdown #1216

merged 5 commits into from
May 24, 2024

Commits on May 24, 2024

  1. Use pushdown params to disable time range constraint pushdown 

    The predicate pushdown operations for time range constriants currently
rely on None/not-None state to determine whether or not to push the
ConstrainTimeRange filter node to the source. With the switch to
pushdown params this None/not-None state gets tenuous, as future
updates will need to do things like manage time range constraints,
other time dimension filters, and categorical dimension (and entity)
filters, and relying on externalizing None/not-None combinations for
these various filter types will be challenging.

This change encapsulates the enabling and disabling of time range
constraints inside of PredicatePushdownParams. At the moment, in
order to maintain the existing behavior, we simply internalize the
None/not-None behavior for time range constraints. This will also
allow us to easily retain pushdown processing for categorical dimensions
even when time constraint filters should not be applied, and
gradually centralize those controls as we streamline the callsites.

There is added complexity to this change because of two things.

1. Time constraint updating is scattered around in the DataflowPlanBuilder
2. Conversion metrics currently do predicate pushdown for time constraints

The first of these will be addressed later.
Since we cannot reliably support general predicate pushdown for conversion
metrics, we need to allow for a time-range-only pushdown operation. Today
this is equivalent to pushdown enabled, but that will change shortly.
    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    4f5b093 View commit details
    Browse the repository at this point in the history

  2. Simplify predicate pushdown state management

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    2668539 View commit details
    Browse the repository at this point in the history

  3. Refine time constraint management for predicate pushdown tracking

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    fb9cc1b View commit details
    Browse the repository at this point in the history

  4. Rename and fixup documentation for predicate input type enumeration

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    cdf405a View commit details
    Browse the repository at this point in the history

  5. Fitlers is not a word

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    9769994 View commit details
    Browse the repository at this point in the history