New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Function localrun quits when using partitioned topic. #2330
Comments
Hi, |
master branch on monday. |
I think the problem is state storage is not enabled in cluster mode by default. I am updating the documentation in #2335 . I will verify local run and update here. |
Topic of plat.correctness.verification is a partitioned topic with six partitions.
bin/pulsar-admin functions localrun --jar target/function-starter-1.0.0-all.jar --className com.codelipenghui.pulsar.function.WordCountFunction --inputs persistent://public/default/plat.correctness.verification-partition-0 --output persistent://public/default/plat.correctness.verification.wordcount.output --name plat-correctness-verification-wordcount
bin/pulsar-admin functions localrun --jar target/function-starter-1.0.0-all.jar --className com.codelipenghui.pulsar.function.WordCountFunction --inputs persistent://public/default/plat.correctness.verification --output persistent://public/default/plat.correctness.verification.wordcount.output --name plat-correctness-verification-wordcount Log for quit [root@qa-5-170 /opt/apache-pulsar-2.2.0-incubating-SNAPSHOT]# bin/pulsar-admin functions localrun --jar target/function-starter-1.0.0-all.jar --className com.codelipenghui.pulsar.function.WordCountFunction --inputs persistent://public/default/plat.correctness.verification --output persistent://public/default/plat.correctness.verification.wordcount.output --name plat-correctness-verification-wordcount
14:20:24.335 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntimeFactory - Java instance jar location is not defined, using the location defined in system environment : /opt/apache-pulsar-2.2.0-incubating-SNAPSHOT/instances/java-instance.jar
14:20:24.345 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntimeFactory - Python instance file location is not defined using the location defined in system environment : /opt/apache-pulsar-2.2.0-incubating-SNAPSHOT/instances/python-instance/python_instance_main.py
14:20:24.367 [main] INFO org.apache.pulsar.functions.runtime.RuntimeSpawner - RuntimeSpawner starting function plat-correctness-verification-wordcount - 0
14:20:24.383 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - ProcessBuilder starting the process with args java -cp /opt/apache-pulsar-2.2.0-incubating-SNAPSHOT/instances/java-instance.jar -Dpulsar.functions.java.instance.jar=/opt/apache-pulsar-2.2.0-incubating-SNAPSHOT/instances/java-instance.jar -Dlog4j.configurationFile=java_instance_log4j2.yml -Dpulsar.log.dir=/opt/apache-pulsar-2.2.0-incubating-SNAPSHOT/logs/functions -Dpulsar.log.file=plat-correctness-verification-wordcount org.apache.pulsar.functions.runtime.JavaInstanceMain --jar target/function-starter-1.0.0-all.jar --instance_id 0 --function_id 0b81a970-2cf7-43ad-9b88-17ee169cab5b --function_version 5795d1da-3a02-43e5-a004-c4689e678688 --tenant public --namespace default --name plat-correctness-verification-wordcount --function_classname com.codelipenghui.pulsar.function.WordCountFunction --auto_ack true --processing_guarantees ATLEAST_ONCE --pulsar_serviceurl http://localhost:8080/ --use_tls false --tls_allow_insecure false --hostname_verification_enabled false --max_buffered_tuples 1024 --port 38107 --source_type_classname "java.lang.String" --source_subscription_type SHARED --source_topics_serde_classname {"persistent://public/default/plat.correctness.verification":""} --sink_type_classname "java.lang.String" --sink_topic persistent://public/default/plat.correctness.verification.wordcount.output
14:20:24.397 [main] INFO org.apache.pulsar.functions.runtime.ProcessRuntime - Started process successfully
14:20:34.250 [main] INFO org.apache.pulsar.admin.cli.CmdFunctions - RuntimeSpawner quit because of
14:20:34.252 [Thread-3] INFO org.apache.pulsar.admin.cli.CmdFunctions - Shutting down the localrun runtimeSpawner ... |
I had a discussion with @codelipenghui . the original question was around stateful function. stateful function was shipped as a preview feature in 2.1. however it is not fully integrated in cluster mode. so I will be working on that piece in #2335 . However there is a bigger problem around I marked this issue as |
The underlying bug has been fixed at #2346. The fix is included at 2.1.1 and will be included at 2.2.0 release.. |
First i write a demo package by gradle
I got info :
And then i use https://github.com/streamlio/pulsar-functions-java-starter.git
I got the same.
The text was updated successfully, but these errors were encountered: