[source-postgres] Fix issue #6857

[Wenqi Hu (burakku)]

What

Fixes https://github.com/airbytehq/oncall/issues/6857

We have issue in #6857 about the data conversion error

Currently in our data type converter for debezium, we check if the value is null, if it is and column is non-nullable, we return the default value directly without conversion. Otherwise, return null as is.

Two issues here:

we should convert the default value as well then return it
it appears that we found null value in non-nullable column

How

This PR check if the column is nullable, if so, return null as is
if not, we feed the default value to conversion logic and return the result.

Review guide

User Impact

Can this PR be safely reverted and rolled back?

• YES 💚
• NO ❌

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 19, 2025 7:01am

[Wenqi Hu (burakku)]

/bump-version type=patch changelog="Fix issue that column default value did not get converted"

Bump Version job started... Check job output.

✅ Changes applied successfully. (640ffed62459fedea28305b371e1efde449897d2)

[Yue Li (theyueli)]

I understand we need to convert the default value if field.isOptional returns false.

Could you explain why do we replace

DebeziumConverterUtils.convertDefaultValue(field); with getNumberConvertedValue(field.defaultValue()) ?

[Wenqi Hu (burakku)]

Actually you are right, we should use DebeziumConverterUtils. I think we are misuse isOptional as if it's nullable. I will take another look

[Yue Li (theyueli)]

Thanks for the fix... ideally i'm wondering if we could factor out the structure

if (x == null) {
return field.isOptional() ? null : getXXXTypeConverted(field.defaultValue());
}

I was thinking if we could use

DebeziumConverterUtils.convertDefaultValue(); to replace the getXXXTypeConverted call above, but it seems you are getting rid of the Debezium utility... (and I was wondering why in the comment above I posted)