Skip to content

Feat/expr pickle #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
timsaucer wants to merge 18 commits into
base: feat/proto-codecs
Choose a base branch
from
Draft

Feat/expr pickle #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
f3ddec8
feat: pickle support for Expr via PythonLogicalCodec inline encoding
timsaucer May 14, 2026
970ef28
docs(pickle): user guide + Ray example + CI backstop + UDF.name
timsaucer May 14, 2026
001674a
refactor: drop dead state from PythonFunctionScalarUDF
timsaucer May 14, 2026
2056c9b
refactor(codec): use arrow-rs native IPC for schema serialization
timsaucer May 14, 2026
65f9308
docs: strip implementation jargon from Expr pickle docstrings
timsaucer May 14, 2026
0a71079
docs: reframe distributed-expression docs around the user goal
timsaucer May 14, 2026
0c9809e
feat: inline encoding for Python window UDFs
timsaucer May 14, 2026
d71a95d
feat: inline encoding for Python aggregate UDFs
timsaucer May 14, 2026
29b2599
docs: extend the inline-UDF guarantee to aggregate + window UDFs
timsaucer May 14, 2026
bee5e00
docs(distributing-expressions): link to pickle docs, generalize UDF k…
timsaucer May 14, 2026
b6dd989
docs: drop manual pickle.dumps/loads from worker examples
timsaucer May 14, 2026
87473cd
docs: restructure distribution page for the multiple-approach story
timsaucer May 15, 2026
a8297d4
chore: untrack internal design doc that was committed by accident
timsaucer May 15, 2026
8ef9625
refactor: rename MultiColumnWindowUDF -> PythonFunctionWindowUDF
timsaucer May 15, 2026
0e35cc1
docs(distributing-work): drop __main__ guard from example to match si…
timsaucer May 15, 2026
4cb9ef3
feat: per-session toggle for Python UDF inline encoding
timsaucer May 15, 2026
d66917e
docs: pickle module security warning link + move user-facing prose
timsaucer May 15, 2026
e594cbe
docs(ray-example): drop unnecessary UDF registration
timsaucer May 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ env:
jobs:
test-matrix:
runs-on: ubuntu-latest
# Backstop: a hung multiprocessing worker (e.g. during a pickle regression)
# should not block CI longer than this.
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
Expand Down
Loading