Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ClassNotFoundException: com.fasterxml.jackson.annotation.JsonKey due to the old jackson-annotations-2.9.0 #156

Closed
Hubbitus opened this issue Dec 9, 2020 · 5 comments
Closed

ClassNotFoundException: com.fasterxml.jackson.annotation.JsonKey due to the old jackson-annotations-2.9.0 #156

Hubbitus opened this issue Dec 9, 2020 · 5 comments

Comments

@Hubbitus
Copy link

Hubbitus commented Dec 9, 2020 •
edited

In ksqlDB (embedded connect) I've do:

CREATE SOURCE CONNECTOR csv_source WITH (
	'connector.class'              = 'com.github.jcustenborder.kafka.connect.spooldir.SpoolDirCsvSourceConnector',
        'topic'                        = 'test.ksqldb.csv-spooldir',
	'input.path'             	    = '/home/pasha/@Projects/@Experiments/ksqlDB/@EPM-SSDL/Network_Device-Location_mapping/Russia/wired/',
	'finished.path'                = '/home/pasha/@Projects/@Experiments/ksqlDB/@EPM-SSDL/Network_Device-Location_mapping/Russia/wired/_csv_processed',
	'error.path'                   = '/home/pasha/@Projects/@Experiments/ksqlDB/@EPM-SSDL/Network_Device-Location_mapping/Russia/wired/_csv_error',
	'input.file.pattern'           = '.*\.csv$',
	'schema.generation.enabled'    = 'true',
	'csv.first.row.as.header'      = 'true',
	'schema.generation.key.fields' = 'Network_Device'
 );

[2020-12-09 03:09:58,505] [org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:192)] ERROR WorkerConnector{id=CSV_SOURCE} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector:192)
java.lang.NoClassDefFoundError: com/fasterxml/jackson/annotation/JsonKey
        at com.fasterxml.jackson.databind.introspect.JacksonAnnotationIntrospector.hasAsKey(JacksonAnnotationIntrospector.java:1080)
        at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addFields(POJOPropertiesCollector.java:451)
        at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collectAll(POJOPropertiesCollector.java:378)
        at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getJsonValueAccessor(POJOPropertiesCollector.java:227)
        at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.findJsonValueAccessor(BasicBeanDescription.java:258)
        at com.fasterxml.jackson.databind.ser.BasicSerializerFactory.findSerializerByAnnotations(BasicSerializerFactory.java:391)
        at com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:220)
        at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:169)
        at com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1473)
        at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1421)
        at com.fasterxml.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:520)
        at com.fasterxml.jackson.databind.SerializerProvider.findTypedValueSerializer(SerializerProvider.java:798)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:308)
        at com.fasterxml.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3126)
        at com.fasterxml.jackson.core.base.GeneratorBase.writeObject(GeneratorBase.java:388)
        at com.github.jcustenborder.kafka.connect.utils.jackson.SchemaSerializationModule$Serializer.serialize(SchemaSerializationModule.java:183)
        at com.github.jcustenborder.kafka.connect.utils.jackson.SchemaSerializationModule$Serializer.serialize(SchemaSerializationModule.java:179)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
        at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
        at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4485)
        at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3740)
        at com.github.jcustenborder.kafka.connect.spooldir.AbstractSchemaGenerator.<clinit>(AbstractSchemaGenerator.java:51)
        at com.github.jcustenborder.kafka.connect.spooldir.SpoolDirCsvSourceConnector.generator(SpoolDirCsvSourceConnector.java:44)
        at com.github.jcustenborder.kafka.connect.spooldir.AbstractSpoolDirSourceConnector.start(AbstractSpoolDirSourceConnector.java:58)
        at org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:184)
        at org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:209)
        at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:348)
        at org.apache.kafka.connect.runtime.WorkerConnector.doTransitionTo(WorkerConnector.java:331)
        at org.apache.kafka.connect.runtime.WorkerConnector.doRun(WorkerConnector.java:140)
        at org.apache.kafka.connect.runtime.WorkerConnector.run(WorkerConnector.java:117)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.annotation.JsonKey
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 35 more

It looks like updating dependency to jackson-annotations-2.12.0.jar solves the problem.
@vdesabou vdesabou mentioned this issue Dec 10, 2020
Closed
@arijitmazumdar
Copy link

I have faced the similar challenge while implementing CSV file source from control center.

@jcustenborder
Copy link
Owner

Which version of Kafka Connect are you using?

@Hubbitus
Copy link
Author

I use ksqldb embedded kafka connect. How I could obtain its version?

@jcustenborder
Copy link
Owner

jcustenborder commented Dec 30, 2020 •
edited

Does the same issue happen with regular connect? I'm wondering if it's an issue with classpath isolation. Regular connect uses plugin path to load connectors. You set plugin.path=<some directory> each plugin under there gets it's own classpath. Problems like this used to happen in the early days of connect when we didn't have this feature.

@arijitmazumdar
Copy link

Yes, atleast I am getting this issue with connect with confluent 6.0.1.

jcustenborder added a commit that referenced this issue Jan 6, 2021
@jcustenborder
Change parent version to bring in alternative dependencies. fixes #162.
3deb523 
fixes #156.
jcustenborder added a commit that referenced this issue Jan 19, 2021
@jcustenborder
Multiple fixes (#166)
10dcc69 
* Change parent version to bring in alternative dependencies. fixes #162. fixes #156.

* Refactored to store all references to streams and readers in the InputFile class to ensure they are closed. Cleanup policies now will ensure that file handles are closed. Fixes #163

* Added example of copying headers to field names. Fixes #149.

* Schema settings were not being sent to tasks causing the tasks to error. Fixes #157.

* CsvSchemaGenerator was not using the builder to create an instance the CSVReader. Fixes #164.

* Added check when there are blank lines in a file. Skip any blank lines. Fixes #133.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jcustenborder @Hubbitus @arijitmazumdar