Remove heading null chars to prevent load errors #17
Conversation
| @@ -65,4 +65,15 @@ String castStringForce(Object value) { | |||
| if (!(value instanceof String)) return null; | |||
| return (String)value; | |||
| } | |||
|
|
|||
| final Pattern nullCharsPattern = Pattern.compile("^\\x00+$"); | |||
| String removeHeadNullChars(String str) { | |||
There was a problem hiding this comment.
Why removing only heading characters instead of whole text? It does not make sense.
There was a problem hiding this comment.
I thought \0hoge should be hoge. Because I thought it's better to preserve as much data as possible in this phase and allow users to choose whether ignore this data or not.
Why whole text should be dropped? Is that because \0hoge and hoge are not same? How about change to \\0hoge?
| final Pattern nullCharsPattern = Pattern.compile("^\\x00+$"); | ||
| String removeHeadNullChars(String str) { | ||
| if (str == null) return null; | ||
| // return if first char is not \0 |
| if (str == null) return null; | ||
| // return if first char is not \0 | ||
| if (0 != str.indexOf("\0")) return str; | ||
| // return null if all chars are \0 |
| if (0 != str.indexOf("\0")) return str; | ||
| // return null if all chars are \0 | ||
| if (nullCharsPattern.matcher(str).matches()) return null; | ||
| // remove heading \0(s) if some chars follows |
| @@ -65,4 +65,15 @@ String castStringForce(Object value) { | |||
| if (!(value instanceof String)) return null; | |||
| return (String)value; | |||
| } | |||
|
|
|||
| final Pattern nullCharsPattern = Pattern.compile("^\\x00+$"); | |||
| // return null if all chars are \0 | ||
| if (nullCharsPattern.matcher(str).matches()) return null; | ||
| // remove heading \0(s) if some chars follows | ||
| return str.replaceFirst("^\\x00+", ""); |
There was a problem hiding this comment.
- Why not compile this pattern (while above pattern is compiled)?
- Replace first, then check the string is empty (and return null if empty).
| String removeHeadNullChars(String str) { | ||
| if (str == null) return null; | ||
| // return if first char is not \0 | ||
| if (0 != str.indexOf("\0")) return str; |
There was a problem hiding this comment.
Flipped condition expression (invert of str.indexOf("\0") != 0) is a kind of superstition. Never use that.
|
@aamine please |
|
Fixed. |
shimpeko
force-pushed
the
remove_heading_null_chars
branch
from
21c1215
to
6806f7b
Compare
| @@ -65,4 +65,12 @@ String castStringForce(Object value) { | |||
| if (!(value instanceof String)) return null; | |||
| return (String)value; | |||
| } | |||
|
|
|||
| Pattern afterNullCharPattern = Pattern.compile("\\x00.*"); | |||
There was a problem hiding this comment.
[nits] Constants should be upper chars
|
plz merge |
| @@ -65,4 +65,12 @@ String castStringForce(Object value) { | |||
| if (!(value instanceof String)) return null; | |||
| return (String)value; | |||
| } | |||
|
|
|||
| final private Pattern AFTER_NULL_CHAR_PATTERN = Pattern.compile("\\x00.*"); | |||
There was a problem hiding this comment.
はっ…… static！
あとprivateは滅多につけなくてよいです、むしろ何も付けずにpackage privateにするほうがいい（何も付けないとデフォルトでpackage privateになるということは、利点が大きいからデフォルトにしてあるのです一般的に言って）
|
fixed |
Change text operator to remove heading
\0(null char) which cause load error with Redshift.@aamine please review