[FLINK-29000][python] Support python UDF in the SQL Gateway #21725
Conversation
HuangXingBo
changed the title
HuangXingBo
force-pushed
the
FLINK-29000
branch
from
1af7133
to
6f3dc26
Compare
| @@ -1097,6 +1097,12 @@ under the License. | |||
| <version>${project.version}</version> | |||
| <scope>test</scope> | |||
| </dependency> | |||
| <dependency> | |||
There was a problem hiding this comment.
Why add python dependencies here?
There was a problem hiding this comment.
if we don't add the python jar as test dependency, HiveServer2EndpointITCase and HiveServer2EndpointStatementITCase will failed.
| dependencies.add(location); | ||
| } | ||
| } catch (URISyntaxException | ClassNotFoundException e) { | ||
| throw new SqlExecutionException("Failed to find flink-python jar.", e); |
There was a problem hiding this comment.
throw SqlGatewayException. In the SqlGateway#startSqlGateway, we will notice users it's a bug if the type of the exception is not SqlGatewayException.
| final List<URL> dependencies = new ArrayList<>(); | ||
| // add python dependencies by default | ||
| try { | ||
| URL location = | ||
| Class.forName( | ||
| "org.apache.flink.python.PythonFunctionRunner", | ||
| false, | ||
| Thread.currentThread().getContextClassLoader()) | ||
| .getProtectionDomain() | ||
| .getCodeSource() | ||
| .getLocation(); | ||
| if (Paths.get(location.toURI()).toFile().isFile()) { | ||
| dependencies.add(location); | ||
| } | ||
| } catch (URISyntaxException | ClassNotFoundException e) { | ||
| throw new SqlExecutionException("Failed to find flink-python jar.", e); |
There was a problem hiding this comment.
Can we move this code block to DefaultContext#load? I think we will introduce -l/-j comand line paramters in the future.
There was a problem hiding this comment.
If it is the -j approach, I guess that you want to make the loading of python jar an optional behavior?
There was a problem hiding this comment.
I think we can modify the constructor of the DefaultContext to pass the dependencies. During the loading, we always try to find the python dependencies and add it into the dependencies.
…1725) * [FLINK-29000][python] Support python UDF in the SQL Gateway * fix comments * fix checkstyle
…1725) * [FLINK-29000][python] Support python UDF in the SQL Gateway * fix comments * fix checkstyle
What is the purpose of the change
This pull request will support python UDF in the SQL Gateway
Brief change log
Does this pull request potentially affect one of the following parts:
@Public(Evolving): (no)Documentation