Skip to content

egorlitvinenko/clickhouse-input-client

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

clickhouse-jdbc is a JDBC wrapper, which uses Http Client from Apache Http Components to write to Clickhouse.

Http communication is externalized from clickhouse-jdbc and modified a little bit to reduce impact on GC.

It is used in my experiment. Basic usage from there:

ClickhouseInsertBatch insertBatch = new ClickhouseInsertBatch(fromCreateToLoadBenchmark.clickhouseHttp,
                "INSERT INTO test.TEST_DATA_1M_9C_DATE (ID, f1, f2, f3, f4, f5, f6, f7, f8)",
                BATCH_SIZE,
                new BufferPreparedStream(BATCH_SIZE * 1000)
        );

ChLineEventHandler handler = new ChLineEventHandler(insertBatch);

public class ChLineEventHandler implements EventHandler<LineEvent> {

    private final ClickhouseInsertBatch insertBatch;

    public ChLineEventHandler(ClickhouseInsertBatch insertBatch) {
        this.insertBatch = insertBatch;
    }

    @Override
    public void onEvent(LineEvent event, long sequence, boolean endOfBatch) throws Exception {
        if (event.endStream()) {
            insertBatch.endBatching();
        } else if (event.isFinished() && event.isValid()) {
            insertBatch.addBatch(event.values());
            if (insertBatch.readyToBatch()) {
                insertBatch.execute();
            }
        }
    }

}

p.s. If you use this library or clickhouse-jdbc it is important to configure logger properties.

About

Write to Clickhouse directly using Http Stream

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages