Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make source_semantic_models property accessible from a DataflowPlanNode #1218

Merged
merged 5 commits into from
May 24, 2024
Merged

Make source_semantic_models property accessible from a DataflowPlanNode #1218

merged 5 commits into from
May 24, 2024

Commits on May 24, 2024

  1. Make source_semantic_models property accessible from a DataflowPlanNode 

    This change ultimately adds the source_semantic_models property to
the DataflowPlan and adds a hook for enabling access to it from any
arbitrary DataflowPlanNode.

We currently have two use-cases for this, one in the cloud codebase
that needs the semantic model inputs for a dataflow plan, and the
upcoming predicate pushdown evaluation which needs the semantic model
inputs for a given DataflowPlanNode.

An earlier version of this change added the property directly to the
DataflowPlanNode, which would satisfy both use cases above. The issue
with having this property assigned directly to a DataflowPlanNode
is that the property might be considered both a node-level and graph-level
attribute, so it's not clear where to put the accessor.

The solution we came up with for this was to allow access to a DataflowPlan
DAG object built from the node, which would effectively encapsulate the
subgraph represented by the node and its ancestors. Then we can access
these subgraph properties through the DataflowPlan while making it clear
to the caller that what they are asking for is a subgraph-level, rather
than a node-level attribute.
    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    ec6e037 View commit details
    Browse the repository at this point in the history

  2. Add changelog

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    3dbb0e9 View commit details
    Browse the repository at this point in the history

  3. Rename subgraph helper method

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    4adcbae View commit details
    Browse the repository at this point in the history

  4. Make helper method static

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    18662b0 View commit details
    Browse the repository at this point in the history

  5. Namespace dataflow plan subgraphs by prefix

    @tlento
    tlento committed May 24, 2024
    Configuration menu
    Copy the full SHA
    f0c6cb9 View commit details
    Browse the repository at this point in the history