[TRAFODION-2470] Fixed one cause of 6004 warnings #943
Conversation
Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/1589/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Looks good to me. IMHO we could (and maybe should) restrict unsigned largeint literals more, but that would probably be the subject of another JIRA.
core/sql/parser/SqlParserAux.cpp
Outdated
@@ -863,8 +863,7 @@ ItemExpr *literalOfNumericPassingScale(NAString *strptr, char sign, | |||
} else if (strSize <= 19) { | |||
datatype = (createSignedDatatype ? REC_BIN64_SIGNED : REC_BIN64_UNSIGNED); | |||
length = sizeof(Int64); | |||
} else if (strSize == 20) { | |||
createSignedDatatype = FALSE; | |||
} else if ((strSize == 20) && (!createSignedDatatype)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is in the surround, but I wonder whether we shouldn't also look at the TRAF_LARGEINT_UNSIGNED_IO CQD and only generate unsigned largeint literals if that CQD is set to ON.
Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/1589/ |
Thanks, @zellerh, for the suggestion. I agree, and it was easy to make the suggested change, so I made it. |
New Check Test Started: https://jenkins.esgyn.com/job/Check-PR-master/1590/ |
+1 Thank you for adding this! |
+1 |
Test Passed. https://jenkins.esgyn.com/job/Check-PR-master/1590/ |
The function literalOfNumericPassingScale (parser/SqlParserAux.cpp) is used to parse literals into ConstValue nodes and encoded values. It was incorrectly handling signed numeric literals of 20 characters in length: it would encode these as an unsigned 64-bit integer, then create a signed SQLLargeInt or SQLNumeric with that unsigned encoded value. One symptom of this is that histograms for NUMERIC datatypes would encounter 6004 errors (out-of-order histogram intervals) if there were some values requiring less than 20 characters and others requiring 20 or more.
The fix here reverts to using SQLBigNum instead for such values.