Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sql server support #284

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 62 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# Altinity Replicator for ClickHouse (Lightweight version)
![](doc/img/kafka_replication_tool.jpg)

New tool to replicate data from MySQL, PostgreSQL, MariaDB and Mongo without additional dependencies.
New tool to replicate data from MySQL, PostgreSQL, MariaDB, SQL Server and Mongo without additional dependencies.
Single executable and lightweight.
##### Supports DDL in MySQL.

Expand Down Expand Up @@ -121,6 +121,61 @@ auto.create.tables: "true"
database.dbname: "public"
```

```
### SQL Server Config(sink-connector-lightweight/docker/config_sql_server.yml)
```
```
name: "debezium-embedded-sql-server"
database.hostname: "sql-server-db"
database.port: "1433"
database.user: "sa"
database.password: "Root1234$$"
database.names: "Prime"
table.include.list: "MyTable"
clickhouse.server.url: "clickhouse"
clickhouse.server.user: "root"
clickhouse.server.pass: "root"
clickhouse.server.port: "8123"
clickhouse.server.database: "test"
database.allowPublicKeyRetrieval: "true"
snapshot.mode: "initial"
offset.flush.interval.ms: 5000
connector.class: "io.debezium.connector.sqlserver.SqlServerConnector"
offset.storage: "io.debezium.storage.jdbc.offset.JdbcOffsetBackingStore"
offset.storage.offset.storage.jdbc.offset.table.name: "altinity_sink_connector.replica_source_info"
offset.storage.jdbc.url: "jdbc:clickhouse://clickhouse:8123/altinity_sink_connector"
offset.storage.jdbc.user: "root"
offset.storage.jdbc.password: "root"
offset.storage.offset.storage.jdbc.offset.table.ddl: "CREATE TABLE if not exists %s
(
`id` String,
`offset_key` String,
`offset_val` String,
`record_insert_ts` DateTime,
`record_insert_seq` UInt64,
`_version` UInt64 MATERIALIZED toUnixTimestamp64Nano(now64(9))
)
ENGINE = ReplacingMergeTree(_version)
ORDER BY id
SETTINGS index_granularity = 8198"
offset.storage.offset.storage.jdbc.offset.table.delete: "delete from %s where 1=1"
schema.history.internal: "io.debezium.storage.jdbc.history.JdbcSchemaHistory"
schema.history.internal.jdbc.url: "jdbc:clickhouse://clickhouse:8123/altinity_sink_connector"
schema.history.internal.jdbc.user: "root"
schema.history.internal.jdbc.password: "root"
schema.history.internal.jdbc.schema.history.table.ddl: "CREATE TABLE if not exists %s
(`id` VARCHAR(36) NOT NULL, `history_data` VARCHAR(65000), `history_data_seq` INTEGER, `record_insert_ts` TIMESTAMP NOT NULL, `record_insert_seq` INTEGER NOT NULL) ENGINE=ReplacingMergeTree(record_insert_seq) order by id"

schema.history.internal.jdbc.schema.history.table.name: "altinity_sink_connector.replicate_schema_history"
enable.snapshot.ddl: "true"
auto.create.tables: "true"
database.dbname: "public"
database.ssl.truststore: "${project.basedir}/src/test/resources/ssl"
database.ssl.truststore.password: "debezium"
database.trustServerCertificate: "true"

```

## Command Line(JAR)
https://github.com/Altinity/clickhouse-sink-connector/releases

Expand Down Expand Up @@ -153,6 +208,12 @@ cd sink-connector-lightweight/docker
docker-compose -f docker-compose-postgres-standalone.yml up
```

**SQL Server**
```
cd sink-connector-lightweight/docker
docker-compose -f docker-compose-sql-server.yml up
```

##### Docker
Images are published in Gitlab.

Expand Down
Binary file modified sink-connector-client/sink-connector-client
Binary file not shown.
2 changes: 1 addition & 1 deletion sink-connector-lightweight/build_docker.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash

mvn clean install -DskipTests=true
#mvn clean install -DskipTests=true
today_date=$(date +%F)

docker login registry.gitlab.com
Expand Down

This file was deleted.

Loading
Loading