-
Notifications
You must be signed in to change notification settings - Fork 566
Closed
Description
Description
Add filesystem connector module for inlong metric and inlong audit.
This module add inlong.metric and inlong.audit option for supporting metric computing.
And add sink.ignore.changelog option for supporting changlog stream.
Use case
We can use flink sql to test it.
create table `table_groupId_streamId_nodeId1`(
`age` INT,
`name` STRING,
PRIMARY KEY(`age`) NOT ENFORCED)
WITH (
'connector' = 'mysql-cdc-inlong',
'hostname' = 'xxx',
'username' = 'root',
'password' = 'inlong',
'database-name' = 'test',
'scan.incremental.snapshot.enabled' = 'true',
'server-time-zone' = 'GMT+8',
'table-name' = 'user',
--'inlong.audit' = 'host&port',
'inlong.metric' = 'mysqlGroup&mysqlStream&mysqlnode1'
);
CREATE TABLE `table_groupId_streamId_nodeId2`(
PRIMARY KEY (`name`,`age`) NOT ENFORCED,
`name` STRING,
`age` INT)
WITH (
'connector' = 'filesystem-inlong',
'path' = 'hdfs://xxx:8020/',
'format' = 'csv',
--'inlong.audit' = 'host&port',
'inlong.metric' = 'hdfsgroup&hdfsStream&hdfsNode',
'sink.ignore.changelog' = 'true',
);
INSERT INTO `table_groupId_streamId_nodeId2`
SELECT
`name` AS `name`,
`age` AS `age`
FROM `table_groupId_streamId_nodeId1`;Are you willing to submit PR?
- Yes, I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Reactions are currently unavailable