Skip to content

[INLONG-6507][Sort] Convert date to timestamp in oracle connector#6508

Merged
dockerzhang merged 7 commits intoapache:masterfrom
e-mhui:INLONG-6507
Nov 15, 2022
Merged

[INLONG-6507][Sort] Convert date to timestamp in oracle connector#6508
dockerzhang merged 7 commits intoapache:masterfrom
e-mhui:INLONG-6507

Conversation

@e-mhui
Copy link
Contributor

@e-mhui e-mhui commented Nov 11, 2022

Prepare a Pull Request

[INLONG-6507][Sort] Convert date to timestamp in oracle connector

Motivation

In debezium, the oracle date type will be read as the int64 type, but the oracle date type mapping is the debezium timestamp type, and the debezium timestamp type is the string format of date-time. So, we need to convert the int64 to the string format of date-time.

Modifications

When parsing field value, if the field schema is date or timestamp type, we need to convert the filed value from the number of milliseconds to the string format of date-time. The main logic is as follows:

    /**
     * extract the data with the format provided by debezium
     *
     * @param fieldValue
     * @param schemaName
     * @return the extracted data with schema
     */
    private Object getValueWithSchema(Object fieldValue, String schemaName) {
        if (MicroTime.SCHEMA_NAME.equals(schemaName)) {
            Instant instant = Instant.ofEpochMilli((Long) fieldValue / 1000);
            fieldValue = timeFormatter.format(LocalDateTime.ofInstant(instant, serverTimeZone));
        } else if (Date.SCHEMA_NAME.equals(schemaName)) {
            fieldValue = dateFormatter.format(LocalDate.ofEpochDay((Integer) fieldValue));
        } else if (ZonedTimestamp.SCHEMA_NAME.equals(schemaName)) {
            ZonedDateTime zonedDateTime = ZonedDateTime.parse((CharSequence) fieldValue);
            fieldValue = zonedDateTime.withZoneSameInstant(serverTimeZone).toLocalDateTime()
                    .atZone(ZONE_UTC).format(DateTimeFormatter.ISO_INSTANT);
        } else if (Timestamp.SCHEMA_NAME.equals(schemaName)) {
            Instant instantTime = Instant.ofEpochMilli((Long) fieldValue);
            fieldValue = LocalDateTime.ofInstant(instantTime, ZONE_UTC).toString();
        }
        return fieldValue;
    }

@gong
Copy link
Contributor

gong commented Nov 11, 2022

Maybe, PR description is not consistent with modification of code

Copy link
Member

@EMsnap EMsnap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the mysql connector for the date and timestamp data.

@e-mhui
Copy link
Contributor Author

e-mhui commented Nov 12, 2022

Please check the mysql connector for the date and timestamp data.

Yes, the oracle connector handles date and timestamp data like the mysql connector.

@e-mhui e-mhui requested review from EMsnap and removed request for gong November 12, 2022 03:10
@e-mhui
Copy link
Contributor Author

e-mhui commented Nov 12, 2022

Maybe, PR description is not consistent with modification of code

More details have been added to the PR description.

@gong
Copy link
Contributor

gong commented Nov 12, 2022

Maybe, PR description is not consistent with modification of code

More details have been added to the PR description.

@e-mhui good.

@dockerzhang dockerzhang merged commit 01d6ecd into apache:master Nov 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][Sort] Should convert date type to timestamp type in Oracle connector

6 participants