Further tests for the InlineVerifier #137
Merged
+192
−5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit fb8942f. CONCAT_WS fails if there are two columns where one column is NULL and the other is some data on the source and the order of the above is flipped on the target: mysql> select concat_ws('', md5(1), md5(NULL), md5('hello')); +------------------------------------------------------------------+ | concat_ws('', md5(1), md5(NULL), md5('hello')) | +------------------------------------------------------------------+ | c4ca4238a0b923820dcc509a6f75849b5d41402abc4b2a76b9719d911017c592 | +------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> select concat_ws('', md5(1), md5('hello'), md5(NULL)); +------------------------------------------------------------------+ | concat_ws('', md5(1), md5('hello'), md5(NULL)) | +------------------------------------------------------------------+ | c4ca4238a0b923820dcc509a6f75849b5d41402abc4b2a76b9719d911017c592 | +------------------------------------------------------------------+ 1 row in set (0.00 sec)
Used a magic string to make it much less likely.
To prevent a naive CONCAT_WS introducing a regression.
|
I'm going to merge this for now. If there are any other concerns/comments about this, we can address it in a follow up PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Is a decently straightforward PR, although I do fix an issue with
NULLvs'NULL'.Appendix: Technical analysis of the
NULLvs'NULL'problemOne thing I uncovered is that
NULLis being treated the same as'NULL'by the InlineVerifier. Thus, we're prone to false negatives involving NULL. The cause of the issue is as folllows:MD5function. However,MD5(NULL) -> NULL, andCONCAT(NULL, 'data') -> NULL. This means every row that has a NULL column will have a NULL checksum, which is unacceptable.COALESCEfunction, which returns the first non null entry. Specifically, we usedMD5(COALASCE(column, 'NULL')). If the column is NULL, we would thus computeMD5('NULL'). This causes the collision with the string'NULL'.The current workaround uses a magic string that's unlikely to be found in nature. Although it may be possible to use
CONCAT_WSandNULLIFto accomplish this without the magic string. There are some reverted commits for reference. The commit at the end also adds a test case that will be triggered if the naiveCONCAT_WSmethod is implemented in the future.