feat(python/sedonadb): Make expressions context-aware for piped functions#901
Open
paleolimbot wants to merge 13 commits into
Open
feat(python/sedonadb): Make expressions context-aware for piped functions#901paleolimbot wants to merge 13 commits into
paleolimbot wants to merge 13 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes Python expressions context-aware so that users can use a “piped” syntax (expr.funcs.<fn>(...)) where the expression (or literal) is implicitly passed as the first argument to functions looked up from the owning SedonaContext.
Changes:
- Add
_ctxpropagation toExpr(andLiteral) plus.funcsaccessors to enable context-driven function lookup and piping. - Extend
Functions/ScalarUdf/AggregateUdfto support generating calls with an optional piped first argument. - Update DataFrame column access and tests to validate context propagation, function lookup, and completion helpers.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| python/sedonadb/python/sedonadb/expr/expression.py | Adds _ctx to Expr, introduces .funcs, and implements piped calling for scalar/aggregate UDF wrappers. |
| python/sedonadb/python/sedonadb/expr/literal.py | Adds _ctx to Literal and .funcs to enable piping literals into functions. |
| python/sedonadb/python/sedonadb/functions/init.py | Makes Functions optionally piped from an expression/literal and passes piping metadata into UDF wrappers. |
| python/sedonadb/python/sedonadb/context.py | Plumbs context into col()/lit() and adds _init_from_impl() for wrapper construction. |
| python/sedonadb/python/sedonadb/dataframe.py | Ensures column expressions created from a DataFrame carry a context so .funcs works downstream. |
| python/sedonadb/tests/test_funcs.py | Adds tests for context propagation through Functions when piping an expression. |
| python/sedonadb/tests/expr/test_literal.py | Adds tests for literal piping (literal.funcs.<fn>()) and literal context propagation. |
| python/sedonadb/tests/expr/test_function_expression.py | Removes duplicated funcs accessor tests that moved elsewhere. |
| python/sedonadb/tests/expr/test_expression.py | Adds tests for Expr.funcs and _ctx propagation through expression operations. |
| python/sedonadb/tests/expr/test_dataframe_getitem.py | Asserts DataFrame-derived column Exprs now carry a non-None _ctx. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In #885 we expose scalar and aggregate functions from
sd.funcsbut this still didn't allow for the piped syntax that pandas and polars allow likesd.col("foofy").sum()because there was no way forExprto look up thesumfunction from the context's registry.This PR adds a
_ctxmember on the expression and addsfuncsto get any function from the context (pipingselfin as the first argument).This was extracted from #862 which is a bit more ambitious but not strictly needed (generating source code for inline documentation).