-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NMS-13373: add CortexIo #31
Conversation
be69d56
to
9bfef59
Compare
Implemented storage scheme as supposed in NMS-13374. |
Can we include the storage schema described in https://issues.opennms.org/browse/NMS-13374 as part of the README, or documentation in this project? |
I added a separate |
testing/src/test/java/org/opennms/nephron/testing/flowgen/CortexPaneIT.java
Outdated
Show resolved
Hide resolved
testing/src/test/java/org/opennms/nephron/testing/flowgen/CortexPaneIT.java
Outdated
Show resolved
Hide resolved
@Description("Regular expression for selecting hosts that are included in aggregations for hosts and possibly conversations.") | ||
@Default.String("") | ||
String getCortexConsideredHosts(); | ||
void setCortexConsideredHosts(String value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are the semantics of this filter? "Only persist entries matching this filter out of the top-k over all hosts" vs "Perists the top-k of the hosts matching this filter"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The filter is applied after "topK" calculation. Results stored in ES are not affected.
(In the meantime I found that there is an IPAddressRange
class in OpenNMS. Maybe that should be used to filter hosts instead of a RegEx.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The filter is applied after "topK" calculation. Results stored in ES are not affected.
Makes sense. Follow up question: is this filter matching IPs only or does it match hostnames, too?
Maybe this needs some rewording: including into aggregations sounds like filter before aggregation. Maybe a native speaker could help out?
(In the meantime I found that there is an IPAddressRange class in OpenNMS. Maybe that should be used to filter hosts instead of a RegEx.)
It would. 10.0.1.15 - 10.0.2.12 is super hard to express as an regex.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I switched to using the IPAddressRange machinery of OpenNMS. This now also supports CIDR notation.
The IPAddressRange machinery boils down to using InetAddress.getByName(dottedNotation)
. Therefore, I think that host names can also be used when specifying the filter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The IPAddressRange machinery boils down to using InetAddress.getByName(dottedNotation). Therefore, I think that host names can also be used when specifying the filter.
Not exactly hat I was asking: If I specify a hostname in this parameter, will this hostname be used to match the hostnames that nephron is transporting besides the IPs of a flow?
Hostname resolution is something we should avoid (or at least not talk about) as it is done on the wrong system.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The filter checks the IP addresses of flows. Host names in the cortexConsideredHosts
argument are resolved on the machine where Nephron is started. You are right, that might not always be what is expected. We can prohibit the use of host names if you prefer.
BTW: The flow classification engine that assigns applications to flows also supports host names in its rules. In that case host names are resolved where the classification engine runs. This may also be surprising.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW: The flow classification engine that assigns applications to flows also supports host names in its rules. In that case host names are resolved where the classification engine runs. This may also be surprising.
Doubt. Maybe the constructor allows this. But it is prevented in the frontend.
We can prohibit the use of host names if you prefer.
This is not about what I prefer but what makes most sense. So, let's have a discussion by exchanging arguments. If there are no arguments and this is just an oversight - go ahead and change it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I copied the corresponding address validation from OpenNMS into Nephron. Host names are no more valid.
cortex/src/main/java/org/opennms/nephron/cortex/PaneAccumulator.java
Outdated
Show resolved
Hide resolved
cortex/src/test/java/org/opennms/nephron/cortex/CortexIoPipelineIT.java
Outdated
Show resolved
Hide resolved
cortex/src/test/java/org/opennms/nephron/cortex/CortexIoPipelineIT.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please make sure that the generated code contains the license header, too.
NMS-13452: GC global state of CortexIo write transform
4de91e5
to
9d6a709
Compare
Issues:
This PR
cortexConsideredHosts
) that allow to specify if / what samples for aggregations for hosts are written.