Skip to content

Conversation

@ronensc
Copy link
Collaborator

@ronensc ronensc commented Mar 14, 2022

This PR:

  • Adds fields recent_op_value and recent_count to the extractor output to be used in prometheus counters
  • Adds an integration test between the extractor and prom encoder

It will follow up with PRs to add RawValuesOperation and a filter field to prom encoder config

@ronensc ronensc force-pushed the refactor-extractor-fields branch from 55049f8 to e6babc9 Compare March 14, 2022 15:35
@ronensc ronensc self-assigned this Mar 14, 2022
@codecov-commenter
Copy link

codecov-commenter commented Mar 14, 2022

Codecov Report

Merging #146 (9a07363) into main (dcfdf2d) will increase coverage by 0.58%.
The diff coverage is 95.16%.

@@            Coverage Diff             @@
##             main     #146      +/-   ##
==========================================
+ Coverage   57.78%   58.37%   +0.58%     
==========================================
  Files          51       51              
  Lines        2916     2950      +34     
==========================================
+ Hits         1685     1722      +37     
+ Misses       1117     1116       -1     
+ Partials      114      112       -2     
Flag Coverage Δ
unittests 58.37% <95.16%> (+0.58%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/pipeline/extract/aggregate/aggregate.go 94.95% <93.18%> (-2.05%) ⬇️
pkg/pipeline/encode/encode_prom.go 81.77% <100.00%> (+2.70%) ⬆️
pkg/test/utils.go 82.60% <100.00%> (+3.66%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dcfdf2d...9a07363. Read the comment docs.

@ronensc ronensc force-pushed the refactor-extractor-fields branch from e6babc9 to 14db167 Compare March 15, 2022 08:48
@ronensc ronensc marked this pull request as ready for review March 15, 2022 08:48
@ronensc ronensc requested review from KalmanMeth and eranra March 15, 2022 08:48
Copy link
Collaborator

@eranra eranra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to update the documentation (at least the README.md) to reflect the changes.

// The test sends flows in 2 batches. Each batch is passed through the extractor and the encoder.
// The output of each stage is verified.
// The output of the 2nd batch depends on the 1st batch.
func Test_Extract_Encode(t *testing.T) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice test :-)

Copy link
Collaborator

@eranra eranra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to update the https://github.com/netobserv/flowlogs-pipeline/blob/main/docs/confGenerator.md do to explain when to use recent_op_value and when to use recent_count

Copy link
Collaborator

@eranra eranra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:-)

@ronensc
Copy link
Collaborator Author

ronensc commented Mar 20, 2022

Need to update the https://github.com/netobserv/flowlogs-pipeline/blob/main/docs/confGenerator.md do to explain when to use recent_op_value and when to use recent_count

I'll do that as part of #147 since it already edits the file

@ronensc ronensc merged commit beb3c57 into netobserv:main Mar 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants