Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WorkerSincTask Error 5014 #77

Closed
xiekevin opened this issue Oct 29, 2019 · 9 comments
Closed

WorkerSincTask Error 5014 #77

xiekevin opened this issue Oct 29, 2019 · 9 comments

Comments

@xiekevin
Copy link

xiekevin commented Oct 29, 2019 •
edited 

[kafkaconnect-9e4947f-2]2019-10-29T18:57:52.282205[kafka-connect][2019-10-29 18:57:52,281] ERROR WorkerSinkTask{id=SnowflakeSink-0} Commit of offsets threw an unexpected exception for sequence number 1: null (org.apache.kafka.connect.runtime.WorkerSinkTask:259)
[kafkaconnect-9e4947f-2]2019-10-29T18:57:52.282205[kafka-connect]com.snowflake.kafka.connector.internal.SnowflakeKafkaConnectorException:
[kafkaconnect-9e4947f-2]2019-10-29T18:57:52.282205[kafka-connect][SF_KAFKA_CONNECTOR] Exception: Failed to put records
[kafkaconnect-9e4947f-2]2019-10-29T18:57:52.282205[kafka-connect][SF_KAFKA_CONNECTOR] Error Code: 5014
[kafkaconnect-9e4947f-2]2019-10-29T18:57:52.282205[kafka-connect][SF_KAFKA_CONNECTOR] Detail: SinkTask hasn't been initialized before calling PUT function```

Can anyone help me identify the root cause of this error?
@binglihub
Copy link
Contributor

could you try to remove the snowpipe generated by connector and restart the connector?

@binglihub
Copy link
Contributor

or, just restart connector again

@xiekevin
Copy link
Author

xiekevin commented Oct 29, 2019 •
edited

Ah thanks for the help @binglihub. I've tried restarting the connector many times, but the message is the same. I've tried removing and adding the connector back in as well. I'm using it as part of a managed service for Kafka Connect (Aiven) so the amount of things I can change is fairly limited. There are some logs above the error message which are provided below:

[kafka-connect][SecurityManager-1(126)] INFO net.snowflake.ingest.connection.SecurityManager - Creating Token with issuer XXXX.USER.SHA256:HASH
[kafka-connect][SecurityManager-1(126)] INFO net.snowflake.ingest.connection.SecurityManager - Creating Token with subject XXXX.USER
[kafka-connect][SecurityManager-1(126)] INFO net.snowflake.ingest.connection.SecurityManager - Created new JWT - JWT_TOKEN
[kafka-connect][SF_KAFKA_CONNECTOR] SnowflakeSinkTask:stop (com.snowflake.kafka.connector.SnowflakeSinkTask:139)
[kafka-connect][2019-10-29 20:29:53,095] INFO
[kafka-connect][2019-10-29 20:29:53,094] ERROR WorkerSinkTask{id=SnowflakeSink-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:180)
[kafka-connect]at java.base/java.lang.Thread.run(Thread.java:834)
[kafka-connect]at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[kafka-connect]at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[kafka-connect]at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[kafka-connect]at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:196)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerSinkTask.closePartitions(WorkerSinkTask.java:590)
[kafka-connect]at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:396)
[kafka-connect]at com.snowflake.kafka.connector.SnowflakeSinkTask.close(SnowflakeSinkTask.java:183)
[kafka-connect]at com.snowflake.kafka.connector.SnowflakeSinkTask.getSink(SnowflakeSinkTask.java:94)
[kafka-connect]at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:321)
[kafka-connect]at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:362)
[kafka-connect][SF_KAFKA_CONNECTOR] Detail: SinkTask hasn't been initialized before calling PUT function
[kafka-connect][SF_KAFKA_CONNECTOR] Error Code: 5014
[kafka-connect][SF_KAFKA_CONNECTOR] Exception: Failed to put records
[kafka-connect]com.snowflake.kafka.connector.internal.SnowflakeKafkaConnectorException:
[kafka-connect][2019-10-29 20:29:53,094] ERROR WorkerSinkTask{id=SnowflakeSink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
[kafkaconnect-9e4947f-2]2019-10-29T20:29:53.099094[kafka-connect][2019-10-29 20:29:53,098] INFO [Consumer clientId=connector-consumer-SnowflakeSink-0, groupId=connect-SnowflakeSink] Member connector-consumer-SnowflakeSink-0-9aa72fb1-2261-4084-b945-26bf310c7420 sending LeaveGroup request to coordinator 159.203.99.0:19154 (id: 2147483636 rack: null) (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:879)

Am I not authenticating correctly?

@binglihub
Copy link
Contributor

Let me test it. it could be a server side issue.

@binglihub
Copy link
Contributor

It is because of STOP function in SinkTask Timeout.
Fixed in 8550186

@xiekevin
Copy link
Author

@binglihub thanks for looking into it! do you know when this change / version 0.5.4 will be released?

@binglihub
Copy link
Contributor

@xiekevin it is available on Maven and Confluent Hub now

@ashinohara
Copy link

I have the most recent connector version 0.5.5 and I am still receiving a similar error when running the kafka connector:

[2019-11-19 12:50:59,491] ERROR WorkerSinkTask{id=Snowflake-7} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. (org.apache.kafka.connect.runtime.WorkerSinkTask:558)
com.snowflake.kafka.connector.internal.SnowflakeKafkaConnectorException:
[SF_KAFKA_CONNECTOR] Exception: Failed to put records
[SF_KAFKA_CONNECTOR] Error Code: 5014
[SF_KAFKA_CONNECTOR] Detail: SinkTask hasn't been initialized before calling PUT function
	at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:362)
	at com.snowflake.kafka.connector.internal.SnowflakeErrors.getException(SnowflakeErrors.java:321)
	at com.snowflake.kafka.connector.SnowflakeSinkTask.getSink(SnowflakeSinkTask.java:94)
	at com.snowflake.kafka.connector.SnowflakeSinkTask.put(SnowflakeSinkTask.java:195)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:538)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:321)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

@xiekevin
Copy link
Author

xiekevin commented Nov 20, 2019 via email

@sdomeshok sdomeshok mentioned this issue Jan 9, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ashinohara @xiekevin @binglihub