How doris handle Java UDF?

[eedalong]

Hi guys, great work! I wonder how Apache Doris handle java udf exactly? I know that Spark extract python udf from operator and have a special op to execute python udf, does Doris also goes this way?

[Gabriel39]

Hi @eedalong , in my mind , python UDF in Spark is executed in a bypass process which is independent from the main process and communicate to each other by RPC. In Doris, Remote UDF is still optional for users. And for JAVA UDF, Doris will call a JNI to pass parameters and get results. So I think it's different between JAVA UDF in Doris and python UDF in Spark.

[eedalong]

Great, After detailed research, I think Doris process C++ udf and java udf in the same way right except that java udf is executed through JNI. In Spark, I think python udf in a operator is extracted out from original operator and Spark uses a special BatchEvalPython to execute this udf right?

[Gabriel39]

Not exactly. C++ UDF in Doris is provided in row-based engine, and it is a pure scalar call for each row. On the other hand ,JAVA UDF is only supported in vectorized engine now. For performance consideration, a batch call is used to reduce JNI overhead.
In a nutshell, C++ UDF and JAVA UDF are almost same for users but different ways used to implement