Use separate classloader loading user functions JARs

[jerrypeng]

Motivation

Currently, we use are using a custom shaded java-instance.jar to run functions.

This creates a couple issues:

1. Increases the size of binary distributions
2. We have to keep shading dependencies as they are added to the pulsar function runtime. This can be tricky especially for transitive dependencies
3. We don't shade pulsar dependencies e.g. pulsar-common. If a user decides to use pulsar-client-admin or pulsar-client within a function that won't work due class clashes.

Proposal

A more elegant solution is instead of using a uber clustom shaded JAR to run functions, we can use separate classloaders to load function framework code and user code.

To be more precise, we will have one "root" classloader that will load pulsar-client-api and pulsar-function-api JARs since they contain interfaces that the function framework code needs to communicate/share with the user function code. From this "root" classloader, we will create a child classloader i.e. function framework classloader that will load all the pulsar client impl JARs and function impl JARS. Basically everything under the "lib/" directory. We will create another child classloader from the "root" classloader that will load the user function code.

Since the function code in a separate classloader, users can still use what dependencies of whichever version they choose yet still not conflict with the function framework code.

This eliminates the the need to to have java-instance.jar and users can also call pulsar-client or pulsar-client-admin from within a function.

some sample code to demonstrate this:

 File functionApiJar = new File("~/.m2/repository/org/apache/pulsar/pulsar-functions-api/2.4.0-SNAPSHOT/pulsar-functions-api-2.4.0-SNAPSHOT.jar");
File pulsarClientApiJar = new File("~/.m2/repository/org/apache/pulsar/pulsar-client-api/2.4.0-SNAPSHOT/pulsar-client-api-2.4.0-SNAPSHOT.jar");
// Our root class loader that contains all the interfaces
ClassLoader root = loadJar(ClassLoader.getSystemClassLoader().getParent(), new File[]{functionApiJar, pulsarClientApiJar});

// Our function framework classloader that loads all pulsar internal JARS
File libDir = new File("/tmp/apache-pulsar-2.4.0-SNAPSHOT/lib");
ClassLoader functionFrameworkClsLoader = loadJar(root, libDir.listFiles());

// Our user function classloader that only has the interfaces and user code
File userFunctionJar = new File("~/my_function.jar");
ClassLoader userFunctionClsLoader = loadJar(root, new File[]{userFunctionJar});

Class<?> functionClass = userFunctionClsLoader.loadClass("org.apache.pulsar.TestFunction");
Object obj = createInstance(functionClass.getName(), userFunctionClsLoader);
Method method = obj.getClass().getDeclaredMethod("process", Object.class, userFunctionClsLoader.loadClass("org.apache.pulsar.functions.api.Context")); 

method.invoke(obj, tweetData, null)

[jerrypeng]

@sijie @srkukarni @merlimat please comment. Thanks!

[merlimat]

👍

[sijie]

This proposal looks good to me. +1

[KannarFr]

@jerrypeng I think #4321 is related.

[srkukarni]

+1 from me.