Skip to content

Add Python UDF UI parameter support #5044

@carloea2

Description

@carloea2

Task Summary

Add support for Python UDF authors to declare UI-editable parameters directly in their UDF code using self.UiParameter(...).

Today, users who want configurable Python UDF behavior usually need to hard-code values in the Python editor or manually manage operator properties. This feature lets the Python code declare the expected parameters, while the Texera UI provides input fields for the values.

Example user code:

from pytexera import *
from typing import Iterator, Optional

class ProcessTupleOperator(UDFOperatorV2):

    @overrides
    def open(self):
        self.threshold = self.UiParameter(
            name="threshold",
            type=AttributeType.DOUBLE,
        ).value

        self.label = self.UiParameter(
            name="label",
            type=AttributeType.STRING,
        ).value

    @overrides
    def process_tuple(self, tuple_: Tuple, port: int) -> Iterator[Optional[TupleLike]]:
        if tuple_["score"] >= self.threshold:
            tuple_["label"] = self.label
            yield tuple_

Expected user workflow:

Step User action Texera behavior
1 User opens a Python UDF operator. The Python code editor is shown as usual.
2 User writes self.UiParameter(name=..., type=...) declarations in the UDF class. Texera detects supported parameter declarations.
3 User opens the operator properties panel. Texera shows one row per inferred UI parameter.
4 User edits parameter values in the UI. Parameter names and types stay locked; only values are editable.
5 User runs the workflow. Texera persists the values, injects them into the Python UDF, and returns typed Python values through .value.
6 User changes the Python declarations. Texera updates the inferred parameter structure while preserving matching existing values when possible.

Supported UDF classes:

Python UDF class Supported
ProcessTupleOperator Yes
ProcessBatchOperator Yes
ProcessTableOperator Yes
GenerateOperator Yes

Supported UI parameter types:

Python declaration UI/runtime type
AttributeType.STRING string
AttributeType.INT / AttributeType.INTEGER integer
AttributeType.LONG long
AttributeType.DOUBLE double
AttributeType.BOOL / AttributeType.BOOLEAN boolean
AttributeType.TIMESTAMP timestamp

Unsupported UI parameter types:

Python declaration Behavior
AttributeType.BINARY Rejected / ignored
AttributeType.LARGE_BINARY Rejected / ignored

Reviewer-preferred implementation stack:

PR Scope Goal PR
1 Frontend Add parser, sync, type, and Formly component building blocks. #5043
2 Scala backend Add descriptor model and injection logic for persisted uiParameters.
3 Python backend Add runtime support for declared UI parameters and typed value conversion.
4 End-to-end Wire descriptors, UI, and runtime together and verify workflows.
sequenceDiagram
    autonumber

    actor User

    participant Code as Python UDF Code
    participant FE as Frontend Parser / UI
    participant Props as Operator Properties
    participant Scala as Scala Backend Injector
    participant Runtime as Python Runtime
    participant UDF as User UDF Class

    User->>Code: Declares self.UiParameter(...)
    Code->>FE: Python code changes
    FE->>FE: Infer parameter name + type
    FE->>Props: Expose editable parameter values
    User->>Props: Enters values
    Props->>Scala: Persist uiParameters with operator
    User->>Scala: Runs workflow
    Scala->>Runtime: Inject parameter values into Python code
    Runtime->>UDF: Execute operator
    UDF->>Runtime: Reads typed values via .value
Loading

Acceptance criteria:

Area Criteria
Authoring Users can declare UI parameters in supported Python UDF classes with self.UiParameter(...).
UI behavior Inferred parameter names and types are locked; parameter values are editable.
Value preservation Existing values are preserved when code changes keep the same parameter names.
Runtime behavior Python UDF code receives typed values from self.UiParameter(...).value.
Validation Missing names/types, unsupported argument patterns, and binary/large-binary types are rejected.
Compatibility Existing native operators and Python UDF workflows still execute.
Testing Add focused frontend, Scala, Python, and browser/manual verification for the stacked changes.

Task Type

  • Refactor / Cleanup
  • DevOps / Deployment / CI
  • Testing / QA
  • Documentation
  • Performance
  • Other

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No fields configured for Task.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions