Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Sort] Add metric computing of MySQL and PostgreSQL and HBase for user query metric by label #5072

Closed
2 tasks done
gong opened this issue Jul 15, 2022 · 0 comments · Fixed by #5073
Closed
2 tasks done

Comments

@gong
Copy link
Contributor

gong commented Jul 15, 2022

Description

Add metric computing of MySQL and PostgreSQL and HBase for user query metric by label value groupId or streamId or nodeId

Use case

User only need add with param "inlong.metric":"<groupId>&<streamId>&<nodeId>" when use flink sql.
example:

  create table `table_groupId_streamId_nodeId1`(
    `age` INT,
    `name` STRING,
    PRIMARY KEY(`age`) NOT ENFORCED)
    WITH (
    'connector' = 'postgres-cdc-inlong',
    'hostname' = 'ip',
    'port' = '5432',
    'username' = 'postgres',
    'password' = 'inlong',
    'database-name' = 'postgres',
    'schema-name' = 'public',
    'table-name' = 'user',
     'decoding.plugin.name' = 'pgoutput',
  'slot.name' = 'feaafacbaddadc',
   'inlong.metric' = 'pgGroup&pgStream&pgNode1'
);

CREATE TABLE `table_groupId_streamId_nodeId2`(
    PRIMARY KEY (`name`,`age`) NOT ENFORCED,
    `name` STRING,
    `age` INT)
    WITH (
    'connector' = 'print'
);

   INSERT INTO `table_groupId_streamId_nodeId2`
    SELECT
    `name` AS `name`,
    `age` AS `age`
    FROM `table_groupId_streamId_nodeId1`;

we can set flink-conf.yaml add metric report

metric.reporters: promgateway
metrics.reporter.promgateway.class: org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter
metrics.reporter.promgateway.host: ip
metrics.reporter.promgateway.port: 9091
metrics.reporter.promgateway.interval: 60 SECONDS
metrics.reporter.promgateway.filterLabelValueCharacters: false

we can see metric in pushgateway.

I will add this feature for mysql cdc and postgresql cdc and HBase.
link issue

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

@gong gong changed the title [Feature] [Sort] Add metric computing for user query metric by label value groupId or streamId or nodeId [Feature] [Sort] Add metric computing of MySQL and PostgreSQL and HBase for user query metric by label value groupId or streamId or nodeId Jul 17, 2022
@healchow healchow changed the title [Feature] [Sort] Add metric computing of MySQL and PostgreSQL and HBase for user query metric by label value groupId or streamId or nodeId [Feature][Sort] Add metric computing of MySQL and PostgreSQL and HBase for user query metric by label Jul 19, 2022
@healchow healchow added this to the 1.3.0 milestone Jul 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants