Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[connector-debezium] Allow user to pass other configs for JsonConvert… #615

Merged
merged 2 commits into from
Mar 9, 2022

Conversation

paul8263
Copy link
Contributor

…er in JsonDebeziumDeserializationSchema

This change allows user to add other config for JsonConverter in JsonDebeziumDeserializationSchema.

Background:
By default JsonConverter in org.apache.kafka.connect.json will parse the value typed BigDecimal to Base64 value, as JsonConverterConfig will set 'decimal.format' to 'DecimalFormat.BASE64.name()' as the default value. For example a record with BigDecimal fields will be converter to: {"NAME":"Kate","NUMBER":"FA=="}, which does not meet the requirements. However, currently we cannot control this behavior because there are no configs we can pass to JsonDebeziumDeserializationSchema constructor. This PR is to solve this.

@leonardBang
Copy link
Contributor

Thanks @paul8263 for the contribution, sorry for the late reply, could you add a test and documentation for the change ?

@paul8263 paul8263 force-pushed the json-deser branch 2 times, most recently from 97e758d to 1a9c053 Compare March 8, 2022 06:03
@paul8263
Copy link
Contributor Author

paul8263 commented Mar 8, 2022

Hi @leonardBang
I added a unit test but got an error: https://dev.azure.com/ververica/flink-cdc-connectors/_build/results?buildId=1050&view=logs&j=ccedd2bf-e82a-54b8-b75f-7971f1fbbb9f&t=385827ba-1388-5c8f-6875-4ed5108d031f&l=1964
It's about Postgres.
By the way where should I put the documentation?

@leonardBang
Copy link
Contributor

Hi @leonardBang , I added a unit test but got an error: https://dev.azure.com/ververica/flink-cdc-connectors/_build/results?buildId=1050&view=logs&j=ccedd2bf-e82a-54b8-b75f-7971f1fbbb9f&t=385827ba-1388-5c8f-6875-4ed5108d031f&l=1964 It's about Postgres. By the way where should I put the documentation?

Could you create an issue for test failure ? we can track the failure tests and fix it later. About the document I think you can refer #381

@leonardBang
Copy link
Contributor

Thanks @paul8263 for the contribution, I found the test is blocking thus I help improve the PR a little, please take a more look

@paul8263
Copy link
Contributor Author

paul8263 commented Mar 9, 2022

Hi @leonardBang ,
Thank your very much. I forget to change the extracted method and the entries in customConverterConfigs were wrongly duplicated.

@leonardBang leonardBang merged commit 8002de4 into apache:master Mar 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants