Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to tag events when parse_json fails to parse #2745

Merged
merged 2 commits into from
May 24, 2023

Conversation

kkondaka
Copy link
Collaborator

@kkondaka kkondaka commented May 24, 2023

Description

Add support to tag events when parse_json fails to parse.
Resolves #2744

Adds tags_on_failure new option to parse_json processor. An example, parse_json processor config would be

processor:                                                                                                                                              
     - parse_json:                                                                                                                                        
          source: "log"                                                                                                                                     
          tags_on_failure: ["json_parse_fail", "tag2"]   

Issues Resolved

Resolves #2744

Check List

  • [ X] New functionality includes testing.
  • New functionality has been documented.
    • New functionality has javadoc added
  • [X ] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Krishna Kondaka <krishkdk@amazon.com>
Signed-off-by: Krishna Kondaka <krishkdk@amazon.com>
@codecov
Copy link

codecov bot commented May 24, 2023

Codecov Report

Merging #2745 (c94ac3b) into main (199f015) will increase coverage by 0.17%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #2745      +/-   ##
============================================
+ Coverage     93.72%   93.90%   +0.17%     
- Complexity     2280     2286       +6     
============================================
  Files           264      264              
  Lines          6389     6394       +5     
  Branches        528      528              
============================================
+ Hits           5988     6004      +16     
+ Misses          265      254      -11     
  Partials        136      136              
Impacted Files Coverage Δ
...lugins/processor/parsejson/ParseJsonProcessor.java 88.88% <100.00%> (+2.00%) ⬆️
.../processor/parsejson/ParseJsonProcessorConfig.java 80.00% <100.00%> (+2.22%) ⬆️

... and 4 files with indirect coverage changes

@@ -287,6 +288,24 @@ void test_when_condition_skips_processing_when_evaluates_to_false() {

}

@Test
void test_tags_when_json_parse_fails() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if tags don't exist? Can we add a check to ensure we aren't building tag set will null values?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cmanning09 you mean, I should check to make sure tags are null when no failures? Something like assertThat(parsedEvent.getMetadata().getTags(), equalTo(Set.of())); to be added in all other tests?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that works. Feel free to add this in a follow up PR.

@kkondaka kkondaka merged commit 5e92474 into opensearch-project:main May 24, 2023
26 checks passed
@kkondaka kkondaka deleted the parse-json-tag branch July 13, 2023 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support to tag events when parse_json fails to parse
3 participants