Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] [StarRocks] Unable to create a sink for identifier 'StarRocks'. #6524

Closed
2 of 3 tasks
nvprasad2003 opened this issue Mar 17, 2024 · 2 comments · Fixed by #6540
Closed
2 of 3 tasks

[Bug] [StarRocks] Unable to create a sink for identifier 'StarRocks'. #6524

nvprasad2003 opened this issue Mar 17, 2024 · 2 comments · Fixed by #6540
Labels
bug

Comments

@nvprasad2003
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

seatunnel for starrocks sink not working, getting below error.

2024-03-18 00:04:10,046 INFO [c.h.i.i.Node ] [main] - [172.30.10.91]:5801 [seatunnel-375198] [5.1] Shutting down connection manager...
2024-03-18 00:04:10,048 INFO [c.h.i.i.Node ] [main] - [172.30.10.91]:5801 [seatunnel-375198] [5.1] Shutting down node engine...
2024-03-18 00:04:13,073 INFO [c.h.i.i.NodeExtension ] [main] - [172.30.10.91]:5801 [seatunnel-375198] [5.1] Destroying node NodeExtension.
2024-03-18 00:04:13,074 INFO [c.h.i.i.Node ] [main] - [172.30.10.91]:5801 [seatunnel-375198] [5.1] Hazelcast Shutdown is completed in 3034 ms.
2024-03-18 00:04:13,075 INFO [c.h.c.LifecycleService ] [main] - [172.30.10.91]:5801 [seatunnel-375198] [5.1] [172.30.10.91]:5801 is SHUTDOWN
2024-03-18 00:04:13,075 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed HazelcastInstance ......
2024-03-18 00:04:13,075 ERROR [o.a.s.c.s.SeaTunnel ] [main] -

===============================================================================

2024-03-18 00:04:13,075 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Fatal Error,

2024-03-18 00:04:13,076 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues

2024-03-18 00:04:13,076 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Reason:SeaTunnel job executed failed

2024-03-18 00:04:13,077 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:199)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a sink for identifier 'StarRocks'.
at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:130)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.createSinkAction(MultipleTableJobConfigParser.java:636)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSink(MultipleTableJobConfigParser.java:567)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:193)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:161)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:146)
... 2 more
Caused by: java.lang.NullPointerException
at java.base/java.lang.String.replace(String.java:2143)
at org.apache.seatunnel.connectors.seatunnel.starrocks.sink.StarRocksSinkFactory.createSink(StarRocksSinkFactory.java:92)
at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:127)
... 8 more

2024-03-18 00:04:13,077 ERROR [o.a.s.c.s.SeaTunnel ] [main] -

Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:199)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a sink for identifier 'StarRocks'.
at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:130)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.createSinkAction(MultipleTableJobConfigParser.java:636)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSink(MultipleTableJobConfigParser.java:567)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:193)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:161)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:146)
... 2 more
Caused by: java.lang.NullPointerException
at java.base/java.lang.String.replace(String.java:2143)
at org.apache.seatunnel.connectors.seatunnel.starrocks.sink.StarRocksSinkFactory.createSink(StarRocksSinkFactory.java:92)
at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:127)
... 8 more

SeaTunnel Version

2.3.4

SeaTunnel Config

env {
  parallelism = 2
  job.mode = "BATCH"
}
source {
  Kafka {
    format = json
    topic = "re"
    bootstrap.servers = "10.91.1.83:9092"
    kafka.config = {
      client.id = client_1
      max.poll.records = 5000
      auto.offset.reset = "earliest"
      enable.auto.commit = "false"
    }
  }
}


sink {
  StarRocks {
    nodeUrls = ["172.30.10.91:8030"]
    base-url =  "jdbc:mysql://172.30.10.91:9030/"
    username = "user
"
    password = "password"
    database = "logs"
    table = "re"
    batch_max_rows = 10000
    trip_outer_array = false
    starrocks.config = {
      format = "JSON"
      strict_mode = "false"
    }
  }
}

Running Command

./bin/seatunnel.sh --config ./config/v2.batch.config.template -e local

Error Exception

./bin/seatunnel.sh --config ./config/v2.batch.config.template1 -e local
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2024-03-18 00:08:47,575 WARN  [ommandArgs$MasterTypeValidator] [main] -
******************************************************************************************
-e and --deploy-mode deprecated in 2.3.1, please use -m and --master instead of it
******************************************************************************************
Mar 18, 2024 12:08:47 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/mnt/vol3/apache-seatunnel-2.3.4/config/seatunnel.yaml' from System property 'seatunnel.config'
Mar 18, 2024 12:08:47 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /mnt/vol3/apache-seatunnel-2.3.4/config/seatunnel.yaml
Mar 18, 2024 12:08:47 AM org.apache.seatunnel.engine.common.config.SeaTunnelConfig
INFO: seatunnel.home is /mnt/vol3/apache-seatunnel-2.3.4
Mar 18, 2024 12:08:47 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/mnt/vol3/apache-seatunnel-2.3.4/config/hazelcast.yaml' from System property 'hazelcast.config'
Mar 18, 2024 12:08:47 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /mnt/vol3/apache-seatunnel-2.3.4/config/hazelcast.yaml
Mar 18, 2024 12:08:47 AM com.hazelcast.instance.impl.HazelcastInstanceFactory
WARNING: Hazelcast is starting in a Java modular environment (Java 9 and newer) but without proper access to required Java packages. Use additional Java arguments to provide Hazelcast access to Java internal API. The internal API access is used to get the best performance results. Arguments to be used:
 --add-modules java.se --add-exports java.base/jdk.internal.ref=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.management/sun.management=ALL-UNNAMED --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED
2024-03-18 00:08:48,033 INFO  [c.h.i.AddressPicker           ] [main] - [LOCAL] [seatunnel-650763] [5.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [172.30.10.91]
2024-03-18 00:08:48,057 INFO  [o.a.s.e.s.SeaTunnelServer     ] [main] - SeaTunnel server start...
2024-03-18 00:08:48,058 INFO  [c.h.system                    ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Based on Hazelcast IMDG version: 5.1.0 (20220228 - 21f20e7)
2024-03-18 00:08:48,058 INFO  [c.h.system                    ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Cluster name: seatunnel-650763
2024-03-18 00:08:48,059 INFO  [c.h.system                    ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1]

 _____               _____                             _
/  ___|             |_   _|                           | |
\ `--.   ___   __ _   | |   _   _  _ __   _ __    ___ | |
 `--. \ / _ \ / _` |  | |  | | | || '_ \ | '_ \  / _ \| |
/\__/ /|  __/| (_| |  | |  | |_| || | | || | | ||  __/| |
\____/  \___| \__,_|  \_/   \__,_||_| |_||_| |_| \___||_|


2024-03-18 00:08:48,059 INFO  [c.h.system                    ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Copyright © 2021-2022 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2024-03-18 00:08:48,059 INFO  [c.h.system                    ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following:
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2024-03-18 00:08:48,061 INFO  [c.h.system                    ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2024-03-18 00:08:48,382 INFO  [c.h.s.security                ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see 🔒 security recommendations and the status of current config.
2024-03-18 00:08:48,439 INFO  [c.h.i.i.Node                  ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Using TCP/IP discovery
2024-03-18 00:08:48,440 WARN  [c.h.c.CPSubsystem             ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
Mar 18, 2024 12:08:48 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/mnt/vol3/apache-seatunnel-2.3.4/config/seatunnel.yaml' from System property 'seatunnel.config'
Mar 18, 2024 12:08:48 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /mnt/vol3/apache-seatunnel-2.3.4/config/seatunnel.yaml
Mar 18, 2024 12:08:48 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/mnt/vol3/apache-seatunnel-2.3.4/config/hazelcast.yaml' from System property 'hazelcast.config'
Mar 18, 2024 12:08:48 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /mnt/vol3/apache-seatunnel-2.3.4/config/hazelcast.yaml
2024-03-18 00:08:48,604 WARN  [o.a.s.e.s.TaskExecutionService] [pool-2-thread-1] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2024-03-18 00:08:48,604 INFO  [o.a.s.e.s.TaskExecutionService] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Created new BusWork : 1654228203
2024-03-18 00:08:48,616 WARN  [a.s.e.s.s.s.DefaultSlotService] [hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource manager, will retry later. this address: [172.30.10.91]:5801
2024-03-18 00:08:48,624 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-5-thread-1] - [172.30.10.91]:5801 [seatunnel-650763] [5.1]
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2024-03-18 00:08:48,662 INFO  [c.h.i.d.Diagnostics           ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2024-03-18 00:08:48,668 INFO  [c.h.c.LifecycleService        ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5801 is STARTING
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.hazelcast.internal.networking.nio.SelectorOptimizer (file:/mnt/vol3/apache-seatunnel-2.3.4/starter/seatunnel-starter.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of com.hazelcast.internal.networking.nio.SelectorOptimizer
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-8] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5807 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-11] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5815 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-14] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5809 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-5] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5830 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-14] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5829 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-11] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5817 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-2] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5822 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-3] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5824 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-8] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5819 is added to the blacklist.
2024-03-18 00:08:48,709 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-6] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5828 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-16] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5821 is added to the blacklist.
2024-03-18 00:08:48,709 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-14] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5825 is added to the blacklist.
2024-03-18 00:08:48,709 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-11] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5808 is added to the blacklist.
2024-03-18 00:08:48,710 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-6] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5816 is added to the blacklist.
2024-03-18 00:08:48,710 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-8] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5802 is added to the blacklist.
2024-03-18 00:08:48,708 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-5] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5827 is added to the blacklist.
2024-03-18 00:08:48,710 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-14] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5812 is added to the blacklist.
2024-03-18 00:08:48,710 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-11] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5810 is added to the blacklist.
2024-03-18 00:08:48,710 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-16] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5814 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-15] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5823 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-9] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5805 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-4] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5818 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-1] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5820 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-10] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5803 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-2] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5806 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-7] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5826 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-12] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5813 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-13] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5811 is added to the blacklist.
2024-03-18 00:08:48,711 INFO  [c.h.i.c.i.TcpIpJoiner         ] [hz.main.cached.thread-3] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5804 is added to the blacklist.
2024-03-18 00:08:49,704 INFO  [c.h.i.c.ClusterService        ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1]

Members {size:1, ver:1} [
        Member [172.30.10.91]:5801 - 48e082bb-3a31-41f6-82ee-326e67108686 this
]

2024-03-18 00:08:49,715 INFO  [c.h.c.LifecycleService        ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5801 is STARTED
Mar 18, 2024 12:08:49 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/mnt/vol3/apache-seatunnel-2.3.4/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
Mar 18, 2024 12:08:49 AM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /mnt/vol3/apache-seatunnel-2.3.4/config/hazelcast-client.yaml
2024-03-18 00:08:49,717 INFO  [o.a.s.e.s.CoordinatorService  ] [pool-4-thread-1] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] This node become a new active master node, begin init coordinator service
2024-03-18 00:08:49,772 INFO  [.c.i.s.ClientInvocationService] [main] - hz.client_1 [seatunnel-650763] [5.1] Running with 2 response threads, dynamic=true
2024-03-18 00:08:49,784 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-650763] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2024-03-18 00:08:49,784 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-650763] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2024-03-18 00:08:49,790 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-650763] [5.1] Trying to connect to cluster: seatunnel-650763
2024-03-18 00:08:49,793 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-650763] [5.1] Trying to connect to [172.30.10.91]:5801
2024-03-18 00:08:49,808 INFO  [.p.t.AuthenticationMessageTask] [hz.main.priority-generic-operation.thread-0] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Received auth from Connection[id=1, /172.30.10.91:5801->/172.30.10.91:11501, qualifier=null, endpoint=[172.30.10.91]:11501, remoteUuid=11afafd2-4bf4-46be-8fe7-5554c15230e3, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 11afafd2-4bf4-46be-8fe7-5554c15230e3, client name: hz.client_1, client version: 5.1
2024-03-18 00:08:49,810 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-650763] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2024-03-18 00:08:49,810 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-650763] [5.1] Authenticated with server [172.30.10.91]:5801:48e082bb-3a31-41f6-82ee-326e67108686, server version: 5.1, local address: /172.30.10.91:11501
2024-03-18 00:08:49,812 INFO  [c.h.i.d.Diagnostics           ] [main] - hz.client_1 [seatunnel-650763] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2024-03-18 00:08:49,820 INFO  [c.h.c.i.s.ClientClusterService] [hz.client_1.event-6] - hz.client_1 [seatunnel-650763] [5.1]

Members [1] {
        Member [172.30.10.91]:5801 - 48e082bb-3a31-41f6-82ee-326e67108686
}

2024-03-18 00:08:49,839 INFO  [.c.i.s.ClientStatisticsService] [main] - Client statistics is enabled with period 5 seconds.
2024-03-18 00:08:49,887 INFO  [.ClientJobExecutionEnvironment] [main] - add common jar in plugins :[]
2024-03-18 00:08:49,894 INFO  [.h.i.p.i.PartitionStateManager] [pool-4-thread-1] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Initializing cluster partition table arrangement...
2024-03-18 00:08:49,906 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Loading config file from path: ./config/v2.batch.config.template1
2024-03-18 00:08:49,956 INFO  [o.a.s.c.s.u.ConfigShadeUtils  ] [main] - Load config shade spi: [base64]
2024-03-18 00:08:50,031 INFO  [o.a.s.c.s.u.ConfigBuilder     ] [main] - Parsed config file:
{
    "env" : {
        "parallelism" : 2,
        "job.mode" : "BATCH"
    },
    "source" : [
        {
            "format" : "json",
            "topic" : "a1-reco-logs-k8s",
            "bootstrap.servers" : "10.91.1.83:9092",
            "plugin_name" : "Kafka",
            "kafka.config" : {
                "client.id" : "client_1",
                "max.poll.records" : 5000,
                "auto.offset.reset" : "earliest",
                "enable.auto.commit" : "false"
            }
        }
    ],
    "sink" : [
        {
            "base-url" : "jdbc:mysql://172.30.10.91:9030/",
            "password" : "user1",
            "database" : "signoz_logs",
            "batch_max_rows" : 10000,
            "starrocks.config" : {
                "format" : "JSON",
                "strict_mode" : "false"
            },
            "nodeUrls" : [
                "172.30.10.91:8030"
            ],
            "trip_outer_array" : false,
            "plugin_name" : "StarRocks",
            "table" : "reco5",
            "username" : "user1"
        }
    ]
}

2024-03-18 00:08:50,048 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /mnt/vol3/apache-seatunnel-2.3.4/connectors
2024-03-18 00:08:50,054 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Kafka'} at: file:/mnt/vol3/apache-seatunnel-2.3.4/connectors/connector-kafka-2.3.4.jar
2024-03-18 00:08:50,055 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='StarRocks'} at: file:/mnt/vol3/apache-seatunnel-2.3.4/connectors/connector-starrocks-2.3.4.jar
2024-03-18 00:08:50,061 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sources.
2024-03-18 00:08:50,116 INFO  [o.a.s.a.t.f.FactoryUtil       ] [main] - get the CatalogTable from source Kafka: Kafka.null.null
2024-03-18 00:08:50,126 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSource Plugin from /mnt/vol3/apache-seatunnel-2.3.4/connectors
2024-03-18 00:08:50,132 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Kafka'} at: file:/mnt/vol3/apache-seatunnel-2.3.4/connectors/connector-kafka-2.3.4.jar
2024-03-18 00:08:50,134 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all transforms.
2024-03-18 00:08:50,134 INFO  [p.MultipleTableJobConfigParser] [main] - start generating all sinks.
2024-03-18 00:08:50,139 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Load SeaTunnelSink Plugin from /mnt/vol3/apache-seatunnel-2.3.4/connectors
2024-03-18 00:08:50,140 INFO  [.s.p.d.AbstractPluginDiscovery] [main] - Discovery plugin jar for: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='StarRocks'} at: file:/mnt/vol3/apache-seatunnel-2.3.4/connectors/connector-starrocks-2.3.4.jar
2024-03-18 00:08:50,154 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-650763] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2024-03-18 00:08:50,161 INFO  [c.h.i.s.t.TcpServerConnection ] [hz.main.IO.thread-in-1] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Connection[id=1, /172.30.10.91:5801->/172.30.10.91:11501, qualifier=null, endpoint=[172.30.10.91]:11501, remoteUuid=11afafd2-4bf4-46be-8fe7-5554c15230e3, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2024-03-18 00:08:50,163 INFO  [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-650763] [5.1] Removed connection to endpoint: [172.30.10.91]:5801:48e082bb-3a31-41f6-82ee-326e67108686, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/172.30.10.91:11501->/172.30.10.91:5801}, remoteAddress=[172.30.10.91]:5801, lastReadTime=2024-03-18 00:08:50.009, lastWriteTime=2024-03-18 00:08:49.895, closedTime=2024-03-18 00:08:50.159, connected server version=5.1}
2024-03-18 00:08:50,163 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-650763] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2024-03-18 00:08:50,165 INFO  [c.h.c.i.ClientEndpointManager ] [hz.main.event-2] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /172.30.10.91:5801->/172.30.10.91:11501, qualifier=null, endpoint=[172.30.10.91]:11501, remoteUuid=11afafd2-4bf4-46be-8fe7-5554c15230e3, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=11afafd2-4bf4-46be-8fe7-5554c15230e3, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1710700729804, latest clientAttributes=lastStatisticsCollectionTime=1710700729839,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1710700729794,clientAddress=172.30.10.91,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=0,os.freePhysicalMemorySize=0,os.freeSwapSpaceSize=0,os.maxFileDescriptorCount=0,os.openFileDescriptorCount=0,os.processCpuTime=0,os.systemLoadAverage=6.53,os.totalPhysicalMemorySize=0,os.totalSwapSpaceSize=0,runtime.availableProcessors=16,runtime.freeMemory=206326296,runtime.maxMemory=536870912,runtime.totalMemory=268435456,runtime.uptime=2830,runtime.usedMemory=62109160, labels=[]}
2024-03-18 00:08:50,168 INFO  [c.h.c.LifecycleService        ] [main] - hz.client_1 [seatunnel-650763] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2024-03-18 00:08:50,168 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2024-03-18 00:08:50,168 INFO  [c.h.c.LifecycleService        ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5801 is SHUTTING_DOWN
2024-03-18 00:08:50,173 INFO  [c.h.i.p.i.MigrationManager    ] [hz.main.cached.thread-13] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Shutdown request of Member [172.30.10.91]:5801 - 48e082bb-3a31-41f6-82ee-326e67108686 this is handled
2024-03-18 00:08:50,180 INFO  [c.h.i.i.Node                  ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Shutting down connection manager...
2024-03-18 00:08:50,183 INFO  [c.h.i.i.Node                  ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Shutting down node engine...
2024-03-18 00:08:53,210 INFO  [c.h.i.i.NodeExtension         ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Destroying node NodeExtension.
2024-03-18 00:08:53,211 INFO  [c.h.i.i.Node                  ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] Hazelcast Shutdown is completed in 3037 ms.
2024-03-18 00:08:53,211 INFO  [c.h.c.LifecycleService        ] [main] - [172.30.10.91]:5801 [seatunnel-650763] [5.1] [172.30.10.91]:5801 is SHUTDOWN
2024-03-18 00:08:53,212 INFO  [s.c.s.s.c.ClientExecuteCommand] [main] - Closed HazelcastInstance ......
2024-03-18 00:08:53,212 ERROR [o.a.s.c.s.SeaTunnel           ] [main] -

===============================================================================


2024-03-18 00:08:53,212 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Fatal Error,

2024-03-18 00:08:53,212 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues

2024-03-18 00:08:53,212 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Reason:SeaTunnel job executed failed

2024-03-18 00:08:53,214 ERROR [o.a.s.c.s.SeaTunnel           ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:199)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a sink for identifier 'StarRocks'.
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:130)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.createSinkAction(MultipleTableJobConfigParser.java:636)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSink(MultipleTableJobConfigParser.java:567)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:193)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:161)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:146)
        ... 2 more
Caused by: java.lang.NullPointerException
        at java.base/java.lang.String.replace(String.java:2143)
        at org.apache.seatunnel.connectors.seatunnel.starrocks.sink.StarRocksSinkFactory.createSink(StarRocksSinkFactory.java:92)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:127)
        ... 8 more

2024-03-18 00:08:53,214 ERROR [o.a.s.c.s.SeaTunnel           ] [main] -
===============================================================================



Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:199)
        at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
        at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a sink for identifier 'StarRocks'.
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:130)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.createSinkAction(MultipleTableJobConfigParser.java:636)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parseSink(MultipleTableJobConfigParser.java:567)
        at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:193)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.getLogicalDag(ClientJobExecutionEnvironment.java:88)
        at org.apache.seatunnel.engine.client.job.ClientJobExecutionEnvironment.execute(ClientJobExecutionEnvironment.java:161)
        at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:146)
        ... 2 more
Caused by: java.lang.NullPointerException
        at java.base/java.lang.String.replace(String.java:2143)
        at org.apache.seatunnel.connectors.seatunnel.starrocks.sink.StarRocksSinkFactory.createSink(StarRocksSinkFactory.java:92)
        at org.apache.seatunnel.api.table.factory.FactoryUtil.createAndPrepareSink(FactoryUtil.java:127)
        ... 8 more

Zeta or Flink or Spark Version

Starrocks version is 3.2

Java or Scala Version

java --version
java 11.0.2 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.2+7-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.2+7-LTS, mixed mode)

Screenshots

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@liunaijie
Copy link
Contributor

it's an bug, will fix it.

@liunaijie liunaijie mentioned this issue Mar 19, 2024
Merged
4 tasks
@liunaijie
Copy link
Contributor

@nvprasad2003 You can add schema to your kafka source to solve this issue.
Config looks like :

source {
 Kafka {
   ...
    schema = {
        fields {
            name = "string"
            age = "int"
            .... other columns
        }
    }
}
}

also, can't add result_table_name option

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
2 participants
@liunaijie @nvprasad2003