[Gandiva][UDF] Solutions to register new UDFs dynamically without checking it into arrow repo.

[asfimport]

Hi there,

Recently I'm studying on gandiva and trying to add some UDF. I noted that it's needed to check the UDF implementation into the arrow repo, register the UDF and then build the UDF into precompiled_bitcode lib, right? I'm just wandering that is it possible to register new UDFs dynamically? Say I have the UDF implementation code locally which is not built into the gandiva lib yet, am I able to call some function or other solutions provided by gandiva officially to register and implement it. Thanks in advance.

Reporter: ZMZ91 / @ZMZ91

_{Note: This issue was originally created as ARROW-8313. Please see the migration documentation for further details.}

[asfimport]

Pindikura Ravindra / @pravindra:
This functionality is not supported yet and I'm not aware of anyone working on it.
 
However, I think it shouldn't be very hard to support this. There are two places where gandiva interacts with functions :

1.  Function registry
   ** the gandiva code looks for fns based on a signature (includes name & input types) in a function registry. The lookup function returns the name of the pre-compiled function
   ** This can be enhanced to support multiple registries - allow for registration of additional registried in addition to the static one.
2. Loading pre-compiled modules
   ** At the time of building the module, the engine loads the pre-compiled IR. This will need to also load other registered libraries (either as a string or a file).
     
    
   And, we will need to come up with some scheme to avoid name conflicts - both the names in the registry & the names of the functions.

[asfimport]

ZMZ91 / @ZMZ91:
Thanks @pravindra. So do you guys have any plan to support it? Or any way that I can contribute it?

[asfimport]

Praveen Kumar / @praveenbingo:
@ZMZ91 As of now, we do not plan to actively work on adding this support. Please go ahead if you want to implement this, we will help out in case of road  blocks.

[asfimport]

ZMZ91 / @ZMZ91:
Thanks for your info @praveenbingo.