[INLONG-7958][Sort] Fix MongoDB's schema becomes unordered after extracting the row data

[e-mhui]

Prepare a Pull Request

[INLONG-7958][Sort] Fix MongoDB's schema becomes unordered after extracting the row data

• Fixes [Bug][Sort] MongoDB's schema becomes unordered after extracting the row data #7958

Motivation

1. MongoDB's schema becomes unordered after extracting the row data. The sink (e.g. iceberg) automatically builds the table, and the source schema being shuffled during the synchronization process will result in inconsistent schema order between source and sink.

2. For all database migration, Mognodb CDC does not specify a primary key, so upsert cannot be implemented. Refer to https://ververica.github.io/flink-cdc-connectors/master/content/connectors/mongodb-cdc.html, we should specify _id as the default primary key.

MongoDB’s change event record doesn’t have updated before message. So, we can only convert it to Flink’s UPSERT changelog stream. An upsert stream requires a unique key, so we must declare _id as primary key. We can’t declare other column as primary key, because delete operation does not contain the key and value besides _id and sharding key.

Modifications

1. Use LinkedHashMap instead of HashMap to make the schema ordered.
2. Use _id as the default primary key
3. Refactor the code

[gong]

@e-mhui Why to keep schema order? I suggest that add reason.

[e-mhui]

@e-mhui Why to keep schema order? I suggest that add reason.

The sink (e.g. iceberg) automatically builds the table, and the source schema being shuffled during the synchronization process will result in inconsistent schema order between source and sink.

[e-mhui]

@e-mhui Why to keep schema order? I suggest that add reason.

The reason has been added to the description.

[gong]

@e-mhui Why to keep schema order? I suggest that add reason.

The sink (e.g. iceberg) automatically builds the table, and the source schema being shuffled during the synchronization process will result in inconsistent schema order between source and sink.

look good to me