Connector configuration is invalid and contains the following 1 error(s) Error while validating connector config: Connection to localhost:5434 refused

[ghkdqhrbals]

When i sent configurations to my source connector, I got error bellow.

{
    "error_code": 400,
    "message": "Connector configuration is invalid and contains the following 1 error(s):\nError while validating connector config: Connection to localhost:5434 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.\nYou can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`"
}

Here's my request in json

POST http://localhost:8083/connectors
{
    "name": "inventory-connector",
    "config": {
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "database.hostname": "localhost",
        "database.port": "5434",
        "database.user": "postgres",
        "database.password": "password",
        "database.dbname" : "chat",
        "database.server.name": "dbserver1"
    }
}   

[ghkdqhrbals]

Additionally, Here's my Kafka + Kafka connector + etc docker-compose file

version : '2'

services:

  # -------- API GATEWAY --------
  nginx:
    restart: always
    container_name: nginx
    depends_on:
      - chatting-server-1
      - chatting-server-2
    build:
      context: ./nginx
      dockerfile: Dockerfile
    ports:
      - "8080:80"

  # -------- KAFKA --------
  zookeeper:
    image: confluentinc/cp-zookeeper:7.2.1
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
  kafka1:
    image: confluentinc/cp-kafka:7.2.1
    container_name: kafka1
    ports:
      - "8097:8097"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: EXTERNAL://localhost:8097,INTERNAL://kafka1:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
  kafka2:
    image: confluentinc/cp-kafka:7.2.1
    container_name: kafka2
    ports:
      - "8098:8098"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: EXTERNAL://localhost:8098,INTERNAL://kafka2:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
  kafka3:
    image: confluentinc/cp-kafka:7.2.1
    container_name: kafka3
    ports:
      - "8099:8099"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: EXTERNAL://localhost:8099,INTERNAL://kafka3:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL

  # -------- Chatting Server --------
  chatting-db-1:
    container_name: chatting-db-1
    image: postgres:12-alpine
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=postgres
      - POSTGRES_DB=chat
    expose:
      - "5433" # Publishes 5433 to other containers but NOT to host machine
    ports:
      - "5433:5433"
    volumes:
      - ./backups:/home/backups
    command: -p 5433
  chatting-server-1:
    container_name: chatting-server-1
    build: ./spring-chatting-backend-server
    ports:
      - "8082:8082"
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://chatting-db-1:5433/chat
      - SPRING_DATASOURCE_USERNAME=postgres
      - SPRING_DATASOURCE_PASSWORD=password
      - SPRING_JPA_HIBERNATE_DDL_AUTO=update
      - SERVER_PORT=8082
      - KAFKA_BOOTSTRAP=kafka1:9092,kafka2:9092,kafka3:9092 # 내부포트
    depends_on:
      - kafka1
      - kafka2
      - kafka3
      - chatting-db-1
    restart: always
  chatting-db-2:
    container_name: chatting-db-2
    image: postgres:12-alpine
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=postgres
      - POSTGRES_DB=chat
    expose:
      - "5434" # Publishes 5433 to other containers but NOT to host machine
    ports:
      - "5434:5434"
    volumes:
      - ./backups:/home/backups
    command: -p 5434
  chatting-server-2:
    container_name: chatting-server-2
    build: ./spring-chatting-backend-server
    ports:
      - "8084:8084"
    environment:
      - SPRING_DATASOURCE_URL=jdbc:postgresql://chatting-db-2:5434/chat
      - SPRING_DATASOURCE_USERNAME=postgres
      - SPRING_DATASOURCE_PASSWORD=password
      - SPRING_JPA_HIBERNATE_DDL_AUTO=update
      - SERVER_PORT=8084
      - KAFKA_BOOTSTRAP=kafka1:9092,kafka2:9092,kafka3:9092 # 내부포트
    depends_on:
      - kafka1
      - kafka2
      - kafka3
      - chatting-db-2
    restart: always

  # -------- Authentication Server --------
  auth-db:
    container_name: auth-db
    image: postgres:12-alpine
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=postgres
      - POSTGRES_DB=auth
    expose:
      - "5435" # Publishes 5433 to other containers but NOT to host machine
    ports:
      - "5435:5435"
    volumes:
      - ./backups:/home/backups
    command: -p 5435
  auth-server:
    container_name: auth-server
    build: ./spring-auth-backend-server
    ports:
      - "8085:8085"
    environment:
      - SERVER_PORT=8085
      - SPRING_DATASOURCE_URL=jdbc:postgresql://auth-db:5435/auth
      - SPRING_DATASOURCE_USERNAME=postgres
      - SPRING_DATASOURCE_PASSWORD=password
      - SPRING_JPA_HIBERNATE_DDL_AUTO=update
    depends_on:
      - auth-db
    restart: always

  # -------- kafdrop for visualization --------
  kafdrop:
    image: obsidiandynamics/kafdrop
    restart: "no"
    environment:
      KAFKA_BROKERCONNECT: "kafka1:9092,kafka2:9092,kafka3:9092"
      JVM_OPTS: "-Xms16M -Xmx512M -Xss180K -XX:-TieredCompilation -XX:+UseStringDeduplication -noverify"
    ports:
      - 9000:9000
    depends_on:
      - kafka1
      - kafka2

  # -------- postgres -> kafka source connector --------
  kafka-source-connector:
    image: debezium/connect:1.9
    container_name: postgres-kafka-source-connector
    ports:
      - 8083:8083
    environment:
      CONFIG_STORAGE_TOPIC: my_connect_configs
      OFFSET_STORAGE_TOPIC: my_connect_offsets
      STATUS_STORAGE_TOPIC: my_connect_statuses
      BOOTSTRAP_SERVERS: kafka1:9092,kafka2:9092,kafka3:9092
    depends_on:
      - kafka1
      - kafka2
      - kafka3
      - zookeeper
      - chatting-db-1
      - chatting-db-2

[ghkdqhrbals]

I replace "database.hostname": "localhost" to "database.hostname": "chatting-db-2" which chatting-db-2 is my postgres docker container name.

And I got an new error.

{
    "error_code": 400,
    "message": "Connector configuration is invalid and contains the following 1 error(s):\nPostgres server wal_level property must be \"logical\" but is: replica\nYou can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`"
}

[ghkdqhrbals]

OK. See reference of default value in Postgresql wal_level here https://postgresqlco.nf/doc/en/param/wal_level/

Default value of WAL(Write-Ahead Logging) is replica! And kafka connector need logical wal_level, so we have to change postgres wal_level to logical.

[ghkdqhrbals]

Solved

I change my postgres configuration in docker compose with command: -c wal_level=logical -p 5434 which can change replica default wal_level to logical value.

and After running docker compose, finally i got 200 response!

{
    "name": "inventory-connector",
    "config": {
        "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
        "database.hostname": "chatting-db-2",
        "database.port": "5434",
        "database.user": "postgres",
        "database.password": "password",
        "database.dbname": "chat",
        "database.server.name": "dbserver1",
        "table.include.list": "public.user_table",
        "name": "inventory-connector"
    },
    "tasks": [],
    "type": "source"
}

[ghkdqhrbals]

This is final process.

1. POST to server

POST http://localhost:8080/chat/user
{
    "userId":"aa",
    "userName":"황보규민"
}

2. In docker container log

chatting-server-2         | 2023-01-05 05:55:07.829  INFO 1 --- [ad | producer-1] chatting.chat.web.ChatController         : 메세지 전송 성공 topic=log-user-add, offset=0, partition=2

3. See kafka with Kafdrop