[mysql] Add schema validation for table schema #493 #522
Conversation
luoyuxia
force-pushed
the
validate_schema
branch
from
8544d82 to
232f065
Compare
| public MySqlValidator(Properties dbzProperties) { | ||
| this.dbzProperties = dbzProperties; | ||
| } | ||
|
|
||
| public MySqlValidator(List<String> columns) { | ||
| this.columns = columns; | ||
| } | ||
|
|
||
| public void initDbzProperties(Properties dbzProperties) { | ||
| this.dbzProperties = dbzProperties; | ||
| } | ||
|
|
There was a problem hiding this comment.
Use inner class Builder to build a ``MySqlValidator`
| } | ||
|
|
||
| /** Validate the schemas from MySQL source contains all columns specified in Flink. */ | ||
| public void validateSchema(Map<TableId, TableChanges.TableChange> schemas) { |
There was a problem hiding this comment.
We'd better use private method validateSchema(), and we can expose public MySqlSchema getResolvedMysqlSchema() method, the field MySqlSchema mySqlSchema is validated in validateSchema.
In this way, the validator API is more clean.
| public SplitEnumerator<MySqlSplit, PendingSplitsState> restoreEnumerator( | ||
| SplitEnumeratorContext<MySqlSplit> enumContext, PendingSplitsState checkpoint) { | ||
| MySqlSourceConfig sourceConfig = configFactory.createConfig(0); | ||
| MySqlValidator validator = new MySqlValidator(sourceConfig.getDbzProperties()); |
There was a problem hiding this comment.
we should skip validation if we're restoring from checkpoint
luoyuxia
force-pushed
the
validate_schema
branch
from
802084f to
ec7537a
Compare
|
Hi @luoyuxia, thanks for your contribution! Could you please rebase this PR with latest |
|
This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. |
|
This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. |
|
This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs. |
|
This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request. |
3 participants
No description provided.