[FLINK-32512] Don't register resource to user resource manager when creating temporary function

[FangYongs]

What is the purpose of the change

When flink perform create temporary function or create system function with a jar file, it will register the jar file to ResourceManager and add it to the ClassLoader. This cause the SHOW JARS statement can list the jar files for these functions, and even when a job does not use these function, the jar file will be aded to the job's classpath.

This PR aims to address this issue with three steps:

1. Create temporary or system function
   a> Download the jar file from remote as needed, which is the same as before for these functions.
   b> Add jar files for temporary and system functions to a new class FunctionResourceManager in ResourceManager when they are created.
   c> Create a new classloader with the added jar files and validate the function class
2. Job uses temporary or system function
   a> Register the jar files for the functions to ResourceManager, which will be added to the classpath of the job
   b> Add the jar files to the classloader for the job
3. Drop temporary or system function
   a> There is reference counter in FunctionResourceManager for each jar file
   b> Register or unregister function to the FunctionResourceManager will update the reference counter
   c> The jar file in FunctionResourceManager will be removed when the reference counter comes to 0

In this way, the jar files for temporary or system function will not be listed in SHOW JARS and will not be added to the classpath and classloader when a job does not use it.

Brief change log

• Added FunctionResourceManager to register and unregister jar files for temporary and system functions
• Don't register jar files to resourceInfos and userClassLoader when temporary and system functions are created, register jar files to FunctionResourceManager
• Register jar files to resourceInfos and userClassLoader when the functions are used in a job

Verifying this change

This change added tests and can be verified as follows:

• Added SHOW JARS; in function.q after creating a temporary functions and the result is empty
• Updated FunctionITCase.testCreateTemporarySystemFunctionByUsingJar to show jars after creating system function
• Added ResourceManagerTest.testRegisterFunctionResource to verify the jar file will no be added to classloader when it is registered in FunctionResourceManager
• Added ResourceManagerTest.testRegisterFunctionResource to check the jar file can be registered and unregistered in FunctionResourceManager with reference counter

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (yes / no) no
• The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no) no
• The serializers: (yes / no / don't know) no
• The runtime per-record code paths (performance sensitive): (yes / no / don't know) no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) no
• The S3 file system connector: (yes / no / don't know) no

Documentation

• Does this pull request introduce a new feature? (yes / no) no
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[flinkbot]

CI report:

• 418eb19 UNKNOWN
• 2136e23 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[FangYongs]

Hi @fsk119 @lsyldliu Please help to review this PR when you're free, thanks

[fsk119]

Thanks for your contribution.

[FangYongs]

Hi @fsk119 , I have updated this PR, please help to review it again, thanks

[KarmaGYZ]

Thanks for the PR, @FangYongs . Please rebase it on the latest master and see my comments.

[FangYongs]

@fsk119 @KarmaGYZ I have updated this PR, please have a look again, thanks

[KarmaGYZ]

Thanks for the update. LGTM except for 2 minor comments.

[fsk119]

LGTM

[KarmaGYZ]

@flinkbot run azure