Skip to content

[Bug] Documented CDC example fails #23662

@AlvaroStream

Description

@AlvaroStream

Search before asking

  • I searched in the issues and found nothing similar.

Read release policy

  • I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

MAC OS 14.7 - Apache 4.0.0 standalone - pulsar-io-debezium-postgres-4.0.0.nar

Minimal reproduce step

As documented here:

Started the container:

ocker run -d -it --rm \                
    --name pulsar-postgres \
    -p 5432:5432 \
    -e POSTGRES_PASSWORD=changeme \
    postgres:13.3 -c wal_level=logical

Started the standalone

./bin/pulsar standalone 

Launched the local run, copied previously the nar in connector directory.


bin/pulsar-admin source localrun \  
    --archive $PWD/connectors/pulsar-io-debezium-postgres-4.0.0.nar \
    --name debezium-postgres-source \
    --tenant public \
    --namespace default \
    --source-config '{"database.hostname": "localhost","database.port": "5432","database.user": "postgres","database.password": "changeme","database.dbname": "postgres","database.server.name": "dbserver1","plugin.name": "pgoutput","schema.whitelist": "public","table.whitelist": "public.users","pulsar.service.url": "pulsar://127.0.0.1:6650"}'

Execute the changes

UPDATE users SET hash_firstname='maxim' WHERE id=1;

and I can see this in the logs of the connection working:

2024-11-29T10:06:53,848+0100 [debezium-postgresconnector-dbserver1-change-event-source-coordinator] INFO  io.debezium.connector.postgresql.connection.AbstractMessageDecoder - Streaming requested from LSN LSN{0/15F8A00}, received LSN LSN{0/15F8A00} identified as already p
rocessed                                                                                                                                                                                                                                                                       
2024-11-29T10:06:53,848+0100 [debezium-postgresconnector-dbserver1-change-event-source-coordinator] INFO  io.debezium.connector.postgresql.connection.WalPositionLocator - Message with LSN 'LSN{0/15F8AA8}' arrived, switching off the filtering                              
2024-11-29T10:07:37,861+0100 [pulsar-client-io-1-3] INFO  org.apache.pulsar.io.kafka.connect.AbstractKafkaConnectSource - Finished flushing offsets to storage                                                 

But the subscription never receives any message:

bin/pulsar-client consume -s "sub-users" public/default/dbserver1.public.users -n 0 -p Earliest  

What did you expect to see?

A message in the topic

What did you see instead?

Nothing, seems the connector is connecting properly to the database

Anything else?

I'm not sure how to debug it more.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugThe PR fixed a bug or issue reported a bug

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions