You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are able to insert data to sink(Postgres) using the Kafka-connect elasticsearch-source from a data source(elastic search).
I have a use case to delete a document from elastic search and the same should be reflected in the data sink(Postgres).
The text was updated successfully, but these errors were encountered:
shanthkumar079
changed the title
Is there any configuration properties in source connector to notify the delete of es_document. in Kafka topic.
Is there any configuration propertie in source connector to notify the delete of es_document. in Kafka topic.
Jun 2, 2021
shanthkumar079
changed the title
Is there any configuration propertie in source connector to notify the delete of es_document. in Kafka topic.
Is there any configuration property in source connector to notify the delete of es document. in Kafka topic.
Jun 3, 2021
Hi @shanthkumar079,
No, it is not possible to detect deletion in elasticsearch using this connector.
However, I suggest to add a new boolean field (e.g. deleted=true/false) to logically delete records in elastic and use that field in postgres to discard logically deleted data. In addition to this you may delete from postgres old data based on retention based.
Hi @DarioBalinzo,
Thanks for your response:-)
I was trying to update a document(value for some fields) by the document Id, seems like this connector doesn't support the update as well.
Please correct me, if my understanding is wrong.
We are able to insert data to sink(Postgres) using the Kafka-connect elasticsearch-source from a data source(elastic search).
I have a use case to delete a document from elastic search and the same should be reflected in the data sink(Postgres).
The text was updated successfully, but these errors were encountered: