Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Maven Central

kafka-connect-hdfs-ext

This project provides extensions for kafka-connect-hdfs project.

Formats

The project provides an additional HDFS connector format: ru.dgis.casino.plain.GzipTextFormat.

GzipTextFormat writes each record value as String separated with \n and also performs compression via GZIPOutputStream.

How to use

Here is an example of a connector config:

connector.class=io.confluent.connect.hdfs.HdfsSinkConnector
format.class=ru.dgis.casino.plain.GzipTextFormat
partitioner.class=io.confluent.connect.hdfs.partitioner.TimeBasedPartitioner
path.format=YYYY/MM/dd
name=SOME_NAME_HERE
topics=TOPIC
hdfs.url=hdfs://YOUR_HADOOP
logs.dir=LOGS_DIR
topics.dir=TOPICS_DIR

flush.size=100000
locale=ru_RU
timezone=Asia/Novosibirsk

About

Set of extensions for kafka connect hdfs

Resources

License

Packages

No packages published