Skip to content

feat(python/sedonadb): Make expressions context-aware for piped functions#901

Open
paleolimbot wants to merge 13 commits into
apache:mainfrom
paleolimbot:expr-ctx
Open

feat(python/sedonadb): Make expressions context-aware for piped functions#901
paleolimbot wants to merge 13 commits into
apache:mainfrom
paleolimbot:expr-ctx

Conversation

@paleolimbot
Copy link
Copy Markdown
Member

@paleolimbot paleolimbot commented Jun 2, 2026

In #885 we expose scalar and aggregate functions from sd.funcs but this still didn't allow for the piped syntax that pandas and polars allow like sd.col("foofy").sum() because there was no way for Expr to look up the sum function from the context's registry.

This PR adds a _ctx member on the expression and adds funcs to get any function from the context (piping self in as the first argument).

This was extracted from #862 which is a bit more ambitious but not strictly needed (generating source code for inline documentation).

import sedona.db

sd = sedona.db.connect()
t = sd.funcs.table.sd_random_geometry()
t.select(geom=t.geometry.funcs.st_buffer(10)).show(2)
# ┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
# │                                               geom                                               │
# │                                             geometry                                             │
# ╞══════════════════════════════════════════════════════════════════════════════════════════════════╡
# │ MULTIPOLYGON(((77.01165169396903 4.571593672098363,77.20379888993672 2.620690451937078,77.77285… │
# ├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
# │ MULTIPOLYGON(((2.2288873806768237 68.82846911756472,2.4210345766445194 66.87756589740344,2.9900… │
# └──────────────────────────────────────────────────────────────────────────────────────────────────┘

@github-actions github-actions Bot requested a review from zhangfengcdt June 2, 2026 18:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes Python expressions context-aware so that users can use a “piped” syntax (expr.funcs.<fn>(...)) where the expression (or literal) is implicitly passed as the first argument to functions looked up from the owning SedonaContext.

Changes:

  • Add _ctx propagation to Expr (and Literal) plus .funcs accessors to enable context-driven function lookup and piping.
  • Extend Functions / ScalarUdf / AggregateUdf to support generating calls with an optional piped first argument.
  • Update DataFrame column access and tests to validate context propagation, function lookup, and completion helpers.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
python/sedonadb/python/sedonadb/expr/expression.py Adds _ctx to Expr, introduces .funcs, and implements piped calling for scalar/aggregate UDF wrappers.
python/sedonadb/python/sedonadb/expr/literal.py Adds _ctx to Literal and .funcs to enable piping literals into functions.
python/sedonadb/python/sedonadb/functions/init.py Makes Functions optionally piped from an expression/literal and passes piping metadata into UDF wrappers.
python/sedonadb/python/sedonadb/context.py Plumbs context into col()/lit() and adds _init_from_impl() for wrapper construction.
python/sedonadb/python/sedonadb/dataframe.py Ensures column expressions created from a DataFrame carry a context so .funcs works downstream.
python/sedonadb/tests/test_funcs.py Adds tests for context propagation through Functions when piping an expression.
python/sedonadb/tests/expr/test_literal.py Adds tests for literal piping (literal.funcs.<fn>()) and literal context propagation.
python/sedonadb/tests/expr/test_function_expression.py Removes duplicated funcs accessor tests that moved elsewhere.
python/sedonadb/tests/expr/test_expression.py Adds tests for Expr.funcs and _ctx propagation through expression operations.
python/sedonadb/tests/expr/test_dataframe_getitem.py Asserts DataFrame-derived column Exprs now carry a non-None _ctx.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread python/sedonadb/python/sedonadb/expr/expression.py Outdated
Comment thread python/sedonadb/python/sedonadb/dataframe.py Outdated
Comment thread python/sedonadb/tests/expr/test_expression.py Outdated
Comment thread python/sedonadb/tests/expr/test_expression.py Outdated
Comment thread python/sedonadb/python/sedonadb/expr/expression.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comment thread python/sedonadb/python/sedonadb/expr/literal.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants