cdc 监控oracle 数据 延迟高

[TYangXiaoChen]

Oracle 数据修改后 cdc获取数据 大概在3～4分钟之间获取到数据 并输出

环境: docker 搭建的 oracle11g 镜像:registry.cn-hangzhou.aliyuncs.com/helowin/oracle_11g
版本 flink-cdc :2.1. flink 版本 2.12 本地环境

[TYangXiaoChen]

package con.firesoon

import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment
import org.apache.flink.streaming.api.functions.source.SourceFunction
import com.ververica.cdc.debezium.{DebeziumSourceFunction, JsonDebeziumDeserializationSchema}
import com.ververica.cdc.connectors.oracle.OracleSource
import com.ververica.cdc.connectors.oracle.table.StartupOptions

import java.util.Properties

/**

• @author Yangxiaochen

• @Date 15/11/2021 17:46

• @Version 1.0
  */
  object OracleSourceExample {
  def main(args: Array[String]): Unit = {
  val properties = new Properties()
  properties.put("database.tablename.case.insensitive","false")
  val source: DebeziumSourceFunction[String] = OracleSource.builderString
  .hostname("172.16.16.242")
  .port(1521)
  .database("helowin") // monitor XE database
  .schemaList("FLINKUSER") // monitor inventory schema
  .tableList("FLINKUSER.DIFF_INFO") // monitor products table
  .username("flinkuser")
  .password("flinkpw")
  //.startupOptions(StartupOptions.latest())
  .debeziumProperties(properties)
  .deserializer(new JsonDebeziumDeserializationSchema()) // converts SourceRecord to JSON String
  .build();

  val env = StreamExecutionEnvironment.getExecutionEnvironment()
  env
  .addSource(source)
  .print().setParallelism(1); // use parallelism 1 for sink to keep message ordering

  env.execute();
  }
  }

[A-little-bit-of-data]

确实延迟比较高，我是用的sql client

[mustang2247]

同上

[Rowen110]

环境说明：flink1.13.1、oracle11g

flinksql cdc client 也发现数据延迟比较大

[gtk96]

我直接使用debezium 采集也是有延迟的。

[star-brilliant]

同上

[dennissky]

https://github.com/ververica/flink-cdc-connectors/wiki/FAQ(ZH)#q1-oracle-cdc-%E7%9A%84%E5%BD%92%E6%A1%A3%E6%97%A5%E5%BF%97%E5%A2%9E%E9%95%BF%E5%BE%88%E5%BF%AB%E4%B8%94%E8%AF%BB%E5%8F%96-log-%E6%85%A2

[leonardBang]

Could you use English for better communication? feel free to reopen once updated the description.