Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Key value processor enhancements #4521

Merged
merged 7 commits into from
May 13, 2024

Conversation

kkondaka
Copy link
Collaborator

@kkondaka kkondaka commented May 9, 2024

Description

Two new config options are added

  1. value_grouping: true/false
    This option groups the values when values start with "(" "[" "<" "{" , single quote, double quote and "http://" and "https://"
    When a value starts with one of the above mentioned strings, all the characters after that are considered part of the value until the corresponding end character is found (ie no keyvalue or field value parsing happens). If the corresponding end character is not found, an error is displayed. When this option is enabled, field_delimiter_regex option is not allowed. All other existing options are valid with this config option.

  2. drop_keys_with_no_value:true/false
    This option drops keys that do not have a value associated with them. This is done after default_values option is processed. Even after default values are assigned, if keys have null value, they are dropped when this config option is set to true

Issues Resolved

Resolves #4447

Check List

  • [X ] New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • [X ] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Krishna Kondaka added 6 commits May 6, 2024 06:19
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
graytaylor0
graytaylor0 previously approved these changes May 10, 2024
@JsonProperty("key_value_when")
private String keyValueWhen;

@AssertTrue(message = "Invalid Configuration. valueGrouping option and field_delimiter_regex are mutually exclusive")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: value_grouping

@@ -496,6 +586,9 @@ private Map<String, Object> executeConfigs(Map<String, Object> map) {
processed.put(pair.getKey(), pair.getValue());
}

if (keyValueProcessorConfig.getDropKeysWithNoValue()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there may not be unit tests for this


public int findInStartGroup(final String str, int idx) {
int j = 0;
while (j < startGroupStrings.length) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there reason not to use for loop?

@kkondaka kkondaka closed this May 13, 2024
@kkondaka kkondaka deleted the key-value-enhancements branch May 13, 2024 05:50
@kkondaka kkondaka restored the key-value-enhancements branch May 13, 2024 14:40
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
@kkondaka kkondaka merged commit 8040fb0 into opensearch-project:main May 13, 2024
89 of 92 checks passed
@kkondaka kkondaka added this to the v2.8 milestone May 14, 2024
@kkondaka kkondaka deleted the key-value-enhancements branch July 30, 2024 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhance Key Value Processor
3 participants