Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug][Sort] Start sort task failed as parsing the config error #3054

Closed
1 of 2 tasks
healchow opened this issue Mar 10, 2022 · 1 comment
Closed
1 of 2 tasks

[Bug][Sort] Start sort task failed as parsing the config error #3054

healchow opened this issue Mar 10, 2022 · 1 comment
Assignees
Labels
type/bug Something is wrong
Milestone

Comments

@healchow
Copy link
Member

What happened

The error log in Flink Job Manager:

2022-03-11 02:00:31,860 WARN  org.apache.flink.runtime.taskmanager.Task                    [] - Source: Pulsar source (1/1)#661 (c71c4057ed239170759f4775fa43c6bb) switched from INITIALIZING to FAILED with failure cause: org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.apache.inlong.sort.protocol.transformation.TransformationRule` (no Creators, like default constructor, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
 at [Source: (byte[])"{"id":3,"source_info":{"type":"pulsar","admin_url":"http://127.0.0.1:8080","service_url":"pulsar://127.0.0.1:6650","topic":"persistent://public/b_test_group/test_stream","subscription_name":"inlong_app_test_stream_consumer_group","deserialization_info":{"type":"json"},"fields":[{"type":"base","name":"dt","format_info":{"type":"timestamp","format":"MILLIS"}},{"type":"base","name":"age","format_info":{"type":"int"}},{"type":"base","name":"name","format_info":{"type":"string"}}],"authentica"[truncated 957 bytes]; line: 1, column: 1131] (through reference chain: org.apache.inlong.sort.protocol.DataFlowInfo["transformation_info"]->org.apache.inlong.sort.protocol.transformation.TransformationInfo["transform_rule"])
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
	...
	at org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3609)
	at org.apache.inlong.sort.meta.zookeeper.ZookeeperMetaWatcher$DataFlowsChildrenWatcherListener.getDataFlowInfo(ZookeeperMetaWatcher.java:158)
	at org.apache.inlong.sort.meta.zookeeper.ZookeeperMetaWatcher$DataFlowsChildrenWatcherListener.onChildAdded(ZookeeperMetaWatcher.java:107)
	at org.apache.inlong.sort.meta.zookeeper.ZookeeperMetaWatcher$DataFlowsChildrenWatcherListener.onInitialized(ZookeeperMetaWatcher.java:149)
	at org.apache.inlong.sort.meta.zookeeper.ZookeeperWatcherUtils.open(ZookeeperWatcherUtils.java:94)
	at org.apache.inlong.sort.meta.zookeeper.ZookeeperMetaWatcher.open(ZookeeperMetaWatcher.java:60)
	at org.apache.inlong.sort.meta.MetaManager.open(MetaManager.java:100)
	at org.apache.inlong.sort.meta.MetaManager.getInstance(MetaManager.java:66)
	at org.apache.inlong.sort.flink.multitenant.pulsar.MultiTopicPulsarSourceFunction.open(MultiTopicPulsarSourceFunction.java:66)

What you expected to happen

The Sort task should be started successfully.

How to reproduce

Push Hive sink info to ZooKeeper and start the Sort job, you'll see the above error.

Environment

No response

InLong version

master

InLong Component

InLong Sort

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

@healchow healchow added the type/bug Something is wrong label Mar 10, 2022
@healchow
Copy link
Member Author

Fixed by #3056.

@healchow healchow self-assigned this Mar 12, 2022
@healchow healchow added this to the 1.1.0 milestone Mar 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Something is wrong
Projects
None yet
Development

No branches or pull requests

1 participant