C++/Rust UDF implementation

[carloea2]

Feature Summary

I feel performance boost can partially be solved by providing faster UDFs. During the hackathon, a previous prototype of a matrix multiply small bench showed C++: 120x , rust: 170x speedups against python.

If it is of interest for apache/texera I am unsure, but it is worth discussing.

Thanks

Proposed Solution or Design

C++ UDF

#include <chrono>

class MatrixMultiplyOperator : public texera::UDFOperator {
public:
    texera::TupleOutput process_tuple(const texera::Tuple& tuple, int port) override {
        int trial = tuple.get("trial").as_int();
        int n = tuple.get("matrix_size").as_int();
        long long seed = tuple.get("seed").as_long();

        auto start = std::chrono::high_resolution_clock::now();

        double checksum = 0.0;
        for (int i = 0; i < n; i++) {
            for (int j = 0; j < n; j++) {
                double cell = 0.0;
                for (int k = 0; k < n; k++) {
                    double a = ((seed + trial * 97LL + i * 31LL + k * 17LL) % 1000LL) / 1000.0;
                    double b = ((seed + trial * 53LL + k * 13LL + j * 29LL) % 1000LL) / 1000.0;
                    cell += a * b;
                }
                checksum += cell * ((i + 1) * 0.001 + (j + 1) * 0.0001);
            }
        }

        auto end = std::chrono::high_resolution_clock::now();
        double elapsed_ms = std::chrono::duration<double, std::milli>(end - start).count();

        return { texera::TupleLike{
            texera::Value::string_value("cpp"),
            texera::Value::double_value(checksum),
            texera::Value::double_value(elapsed_ms)
        }};
    }
};

using TexeraUDFOperator = MatrixMultiplyOperator;

Rust UDF

use std::time::Instant;

#[derive(Default)]
struct MatrixMultiplyOperator;

impl texera::UDFOperator for MatrixMultiplyOperator {
    fn process_tuple(
        &mut self,
        tuple: &texera::Tuple,
        _port: i32,
    ) -> Result<texera::TupleOutput, String> {
        let trial = tuple.get_by_name("trial")?.as_int()?;
        let n = tuple.get_by_name("matrix_size")?.as_int()?;
        let seed = tuple.get_by_name("seed")?.as_long()?;

        let start = Instant::now();

        let mut checksum = 0.0;
        for i in 0..n {
            for j in 0..n {
                let mut cell = 0.0;
                for k in 0..n {
                    let a = ((seed + trial as i64 * 97 + i as i64 * 31 + k as i64 * 17) % 1000) as f64 / 1000.0;
                    let b = ((seed + trial as i64 * 53 + k as i64 * 13 + j as i64 * 29) % 1000) as f64 / 1000.0;
                    cell += a * b;
                }
                checksum += cell * (((i + 1) as f64) * 0.001 + ((j + 1) as f64) * 0.0001);
            }
        }

        let elapsed_ms = start.elapsed().as_secs_f64() * 1000.0;

        Ok(vec![vec![
            texera::Value::string_value("rust"),
            texera::Value::double_value(checksum),
            texera::Value::double_value(elapsed_ms),
        ]])
    }
}

type TexeraUDFOperator = MatrixMultiplyOperator;

Affected Area

No response

[carloea2]

@mengw15 @chenlica @Yicong-Huang @Ma77Ball

[chenlica]

Performance is indeed important. There are other considerations, such as 1) requirements on the runtime environment (e.g., C++ compilers on different operating systems), and 2) security. Can you do such an analsyis?

[carloea2]

1. For compilers, it is up to the user (deployers?), as of now for example for python, user must select and fill the python.exe manually in config.

2. For security, texera security model itself state UDFs are out of the scope, so I believe it is not a blocker.

[Yicong-Huang]

Lovely to see rust! I think that's definitely a great add on to UDF supplies. Python is usually the go to language for its ease of use and its large ecosystem, but now with agent, I think it is not too hard to generate rust/c++ operators.

I fully support this idea!

Although it indeed would add complexity to shipping our release. For service deployed/ maintainer, they need to config compilers etc. for users, they would maybe want to use a particular rust/c++ library. @SarahAsad23 's env work is on this track to make it easier to support Python UDF env, we would need something similar to rust and C++ as well.

Anyway, I think benefit is bigger than cost. Can't wait to see your PRs!

[Yicong-Huang]

A small correction, if not mentioned explicitly, speed is important for Texera!

[Yicong-Huang]

One more thing to consider, how can we support tuple by tuple interactions with those UDFs. to be explored.

[Ma77Ball]

I think this could be split into two PRs. I like the overall direction, though it's worth flagging that integrating Rust and C++ compilers, plus a dedicated virtual environment for each, will require considerable effort.

[Yicong-Huang]

I think this could be split into two PRs. I like the overall direction, though it's worth flagging that integrating Rust and C++ compilers, plus a dedicated virtual environment for each, will require considerable effort.

well I anticipate multiple PRs. it's a heavy task, to introduce any new language support.

[Ma77Ball]

I more meant two issues (that is my bad). I don't think one issue should track this.

[carloea2]

Thanks @Yicong-Huang and @Ma77Ball.

The implementation idea I have in mind is not to introduce a full new C++/Rust Amber engine at first. I think that would be too heavy for the initial version. Python has its own runtime under amber/src/main/python because Python support is a broader execution layer: it handles Python workers, environments, imports, Python objects/dataframes, UDF/source operators, etc.

For C++/Rust, I think the first useful step can be smaller: keep Texera’s existing worker/executor architecture, and treat compiled native code as a persistent UDF worker process owned by a normal Texera operator executor.

So the flow would be:

• Texera still creates a normal logical/physical operator.
• The JVM-side executor owns scheduling, schema propagation, ports, buffering, timeout, and result emission.
• On open(), it compiles or reuses a cached native binary.
• It starts one persistent C++/Rust process for that executor.
• It sends the input schema once.
• During execution, it streams tuple/batch/table frames to the process.
• The native process calls the user’s process_tuple, process_batch, or process_table.
• The executor reads typed output rows back and emits normal Texera tuples.
• On close(), it shuts down the native process.

This means C++/Rust would not be replacing Amber or becoming a second execution engine initially. It would be more like a native-code UDF runtime plugged into the existing Texera worker path.

For @Yicong-Huang’s tuple-by-tuple point, I think we can support that directly with the API:

• process_tuple
• process_batch
• process_table

The native process can stay alive across calls, so tuple-by-tuple execution does not require spawning a process per tuple. That also leaves room for stateful native operators later, since the same operator instance can be reused inside the native process.

For @Ma77Ball’s point about compiler/env complexity, I agree. I think the first version should be explicit that this is deployer/local-environment managed. Similar to how Python currently depends on configured Python environments, C++/Rust would depend on configured compiler paths like CXX / RUSTC and supported compiler flags. Native library/package support should probably be future work, not part of the first pass.

So the main tradeoff is:

• Python UDF remains the flexible, general-purpose scripting path.
• C++/Rust UDFs become a focused native-code path for CPU-heavy UDF kernels.
• A full C++/Rust engine can be considered later if we need native source/sink operators, native dependency management, distributed binary deployment, or deeper fault-tolerance/runtime integration.

I think this keeps the first version useful without committing Texera to a full new language engine immediately.

[carloea2]

Any update about this?

[chenlica]

This is an important topic. I will read the details after finishing my current tasks.

[Yicong-Huang]

Sounds good, having a side car process can be a good start point! It is not clear to me how will you send data to the side car though. Are we using arrow IPC? Are we sending tuple by tuple or on a batch-basis?

[carloea2]

Good question. In the closed prototype PR #5078, I did not use Arrow IPC yet. The MVP used a simple typed row-frame protocol over stdin/stdout:

• The JVM executor selected the input columns, then wrote the column count, base64-encoded column names, type tags, and row data.
• Each field was encoded as <type-tag>:<is-null>:<payload>; strings/binary used base64, while numeric/boolean/timestamp values used textual payloads.
• The native side returned typed output rows on stdout, and the JVM parsed/enforced them against the declared output schema.

The prototype supported three execution APIs:

• process_tuple: one-row frames, mainly for simple or low-latency cases.
• process_batch: the default path, accumulating a configurable batch size before sending a frame.
• process_table: collect input and send once on finish for whole-table algorithms.

One thing I would change from #5078 is the lifecycle. The hackathon prototype was intentionally simple and started the compiled executable per flush. For the real sidecar design, I think the executor should compile/reuse the binary in open(), start one persistent native process, send the schema once, stream tuple/batch/table frames to it, and shut it down in close(). In that design, tuple mode does not mean spawning one process per tuple.

So my preference for the first version is batch-basis by default, while still exposing tuple and table APIs. Arrow IPC is a good candidate for a later transport, especially for larger batches, binary-heavy data, or columnar data. I would keep the transport pluggable: start with a simple debuggable framed protocol like the prototype, then add Arrow IPC after the API/lifecycle is settled.