Configurable value replacement on match failure for RegexExtractionFn #2075
Conversation
|
@b-slim Reopened PR, referring to your earlier comment, what property did you think was unneeded? |
|
@jon-wei |
| Assert.assertTrue(extracted.contains("foobar")); | ||
|
|
||
| byte[] cacheKey = extractionFn.getCacheKey(); | ||
| Assert.assertEquals(17, cacheKey.length); |
There was a problem hiding this comment.
@b-slim added byte array value check for cache key
|
@b-slim I think "injective" is fine here, a regex->match extraction can be thought of as a a function. JavascriptExtractionFn also supports the injective property and it doesn't extend FunctionalExtraction. A hypothetical "injective" use case might be a dimension value with various formats that all contain some unique ID, where the regex is used to strip out parts of the dim val that don't provide any unique information beyond the ID substring. |
jon-wei
force-pushed
the
regex_extract
branch
2 times, most recently
from
bb71a89
to
4e2a3ca
Compare
|
@jon-wei ok i got it now, thanks ! |
|
Very excited about this PR! |
| @@ -24,6 +24,7 @@ | |||
| import com.google.common.base.Preconditions; | |||
| import com.google.common.base.Strings; | |||
| import com.metamx.common.StringUtils; | |||
| import org.jboss.netty.util.internal.StringUtil; | |||
There was a problem hiding this comment.
@gianm removed unused import
jon-wei
force-pushed
the
regex_extract
branch
from
4e2a3ca
to
8e67524
Compare
| @JsonProperty("replaceMissingValues") Boolean replaceMissingValues, | ||
| @JsonProperty("replaceMissingValuesWith") String replaceMissingValuesWith, | ||
| @JsonProperty("injective") Boolean injective | ||
| ) |
There was a problem hiding this comment.
@gianm fixed code style
jon-wei
force-pushed
the
regex_extract
branch
from
8e67524
to
464aa32
Compare
| @JsonProperty("expr") String expr, | ||
| @JsonProperty("replaceMissingValues") Boolean replaceMissingValues, | ||
| @JsonProperty("replaceMissingValuesWith") String replaceMissingValuesWith, | ||
| @JsonProperty("injective") Boolean injective |
There was a problem hiding this comment.
"Injective" doesn't seem to be used for anything right now- is it necessary? What should it be used for?
jon-wei
force-pushed
the
regex_extract
branch
from
464aa32
to
cdbeca0
Compare
jon-wei
force-pushed
the
regex_extract
branch
from
cdbeca0
to
d86953c
Compare
|
👍 after travis |
jon-wei
force-pushed
the
regex_extract
branch
from
d86953c
to
c88f75d
Compare
|
👍 |
Configurable value replacement on match failure for RegexExtractionFn
Addresses issue #2064
This adds 3 new parameters to the RegexExtractionFn:
replaceMissingValues: Boolean, if a regex match fails, replace the dimval with a user specified value instead of returning the orignal dimval, default is false
replaceMissingValuesWith: String, replace dimval with this value when regex match fails if replaceMissingValues == true
injective: Boolean, currently unused, but same usage as "injective" property in the JavaScriptExtractionFn and LookupExtractionFn