Skip to content

Quick start guide for any distribution of Apache Kafka

Jump to bottom
averemee-si edited this page Jul 31, 2023 · 7 revisions

Instructions below are with assumptions that you are running multitenant container database (CDB) (The non-CDB architecture was deprecated in Oracle Database 12c. It can be desupported and unavailable in a release after Oracle Database 19c).

  1. Create non-privileged user for accessing data and grant required privileges:
create user C##ORACDC identified by ORACDC
  default tablespace SYSAUX
  temporary tablespace TEMP
  quota unlimited on SYSAUX
CONTAINER=ALL;
alter user C##ORACDC SET CONTAINER_DATA=ALL CONTAINER=CURRENT;
grant
  CREATE SESSION,
  SET CONTAINER,
  SELECT ANY TRANSACTION,
  SELECT ANY DICTIONARY,
  EXECUTE_CATALOG_ROLE,
  LOGMINING
to C##ORACDC
CONTAINER=ALL;

N.B. Grant privileges as shown above - directly for user!

  1. Optional step - create Oracle wallet to store credentials for user C##ORACDC
mkstore -wrl $A2_CDC_HOME/wallets/ -create
mkstore -wrl $A2_CDC_HOME/wallets/ -createCredential <TNS_ALIAS> C##ORACDC
  1. Enable minimal supplemental logging at database level
sqlplus / as sysdba
alter database add supplemental log data;
  1. Enable supplemental logging for required tables, below examples for schema SCOTT (The story behind scott/tiger - the default login/pass for Oracle)
sqlplus scott/tiger
alter table DEPT add supplemental log data (ALL) columns;
alter table EMP add supplemental log data (ALL) columns;
alter table BONUS add supplemental log data (ALL) columns;
alter table SALGRADE add supplemental log data (ALL) columns;
  1. Set valid parameters in $KAFKA_HOME/config/connect-standalone.properties:
....
bootstrap.servers=<KAFKA-BROKER-HOST>:<KAFKA-BROKER-PORT>
....
offset.storage.file.filename=<FILE-NAME-WHERE-CONNECT-WILL-STORE-OFFSET>
....
plugin.path=<PATH-TO-DIRECTORY-WITH-oracdc-XXXX-standalone.jar>
  1. Create oracdc configuration file $KAFKA_HOME/config/oracdc-test01.properties:

6.1) Without Oracle wallet configured

name=oracdc-test01
connector.class=solutions.a2.cdc.oracle.OraCdcLogMinerConnector
tasks.max=1

a2.jdbc.url=jdbc:oracle:thin:@//<ORACLE_HOST>:<ORACLE_PORT>/<ORACLE_SID>
a2.jdbc.username=C##ORACDC
a2.jdbc.password=ORACDC

a2.tmpdir=<PATH-TO-STORE-TEMPORARY-TRANSACTION-INFORMATION>

6.2) With Oracle wallet configured at step 2)

name=oracdc-test01
connector.class=solutions.a2.cdc.oracle.OraCdcLogMinerConnector
tasks.max=1

a2.wallet.location=<PATH-TO-ORACLE_WALLET>
a2.tns.admin=<TNS_ADMIN>
a2.tns.alias=<TNS-ALIAS>

a2.tmpdir=<PATH-TO-STORE-TEMPORARY-TRANSACTION-INFORMATION>
  1. Create oracdc startup script $KAFKA_HOME/bin/oracdc-test01.sh:
#!/bin/sh
KAFKA_HOME=<PATH-TO-TOP-KAFKA-DIRECTORY>
ORACDC_HOME=<PATH-TO-ORACDC-TOP-DIR>
$KAFKA_HOME/bin/connect-standalone.sh $KAFKA_HOME/config/connect-standalone.properties $KAFKA_HOME/config/oracdc-test01.properties
  1. Run script created at previous step

It's much easier to work with Confluent Platform, there are more configuration and tuning options but for start this is enough

Clone this wiki locally