NMS-13373: add CortexIo #31
Conversation
swachter
force-pushed
the
jira/NMS-13373
branch
2 times, most recently
from
be69d56
to
9bfef59
Compare
|
Implemented storage scheme as supposed in NMS-13374. |
|
Can we include the storage schema described in https://issues.opennms.org/browse/NMS-13374 as part of the README, or documentation in this project? |
|
I added a separate |
testing/src/test/java/org/opennms/nephron/testing/flowgen/CortexPaneIT.java
Outdated
Show resolved
Hide resolved
testing/src/test/java/org/opennms/nephron/testing/flowgen/CortexPaneIT.java
Outdated
Show resolved
Hide resolved
| @Description("Regular expression for selecting hosts that are included in aggregations for hosts and possibly conversations.") | ||
| @Default.String("") | ||
| String getCortexConsideredHosts(); | ||
| void setCortexConsideredHosts(String value); |
There was a problem hiding this comment.
What are the semantics of this filter? "Only persist entries matching this filter out of the top-k over all hosts" vs "Perists the top-k of the hosts matching this filter"?
There was a problem hiding this comment.
The filter is applied after "topK" calculation. Results stored in ES are not affected.
(In the meantime I found that there is an IPAddressRange class in OpenNMS. Maybe that should be used to filter hosts instead of a RegEx.)
There was a problem hiding this comment.
The filter is applied after "topK" calculation. Results stored in ES are not affected.
Makes sense. Follow up question: is this filter matching IPs only or does it match hostnames, too?
Maybe this needs some rewording: including into aggregations sounds like filter before aggregation. Maybe a native speaker could help out?
(In the meantime I found that there is an IPAddressRange class in OpenNMS. Maybe that should be used to filter hosts instead of a RegEx.)
It would. 10.0.1.15 - 10.0.2.12 is super hard to express as an regex.
There was a problem hiding this comment.
I switched to using the IPAddressRange machinery of OpenNMS. This now also supports CIDR notation.
The IPAddressRange machinery boils down to using InetAddress.getByName(dottedNotation). Therefore, I think that host names can also be used when specifying the filter.
There was a problem hiding this comment.
The IPAddressRange machinery boils down to using InetAddress.getByName(dottedNotation). Therefore, I think that host names can also be used when specifying the filter.
Not exactly hat I was asking: If I specify a hostname in this parameter, will this hostname be used to match the hostnames that nephron is transporting besides the IPs of a flow?
Hostname resolution is something we should avoid (or at least not talk about) as it is done on the wrong system.
There was a problem hiding this comment.
The filter checks the IP addresses of flows. Host names in the cortexConsideredHosts argument are resolved on the machine where Nephron is started. You are right, that might not always be what is expected. We can prohibit the use of host names if you prefer.
BTW: The flow classification engine that assigns applications to flows also supports host names in its rules. In that case host names are resolved where the classification engine runs. This may also be surprising.
There was a problem hiding this comment.
BTW: The flow classification engine that assigns applications to flows also supports host names in its rules. In that case host names are resolved where the classification engine runs. This may also be surprising.
Doubt. Maybe the constructor allows this. But it is prevented in the frontend.
We can prohibit the use of host names if you prefer.
This is not about what I prefer but what makes most sense. So, let's have a discussion by exchanging arguments. If there are no arguments and this is just an oversight - go ahead and change it.
There was a problem hiding this comment.
I copied the corresponding address validation from OpenNMS into Nephron. Host names are no more valid.
cortex/src/main/java/org/opennms/nephron/cortex/PaneAccumulator.java
Outdated
Show resolved
Hide resolved
cortex/src/test/java/org/opennms/nephron/cortex/CortexIoPipelineIT.java
Outdated
Show resolved
Hide resolved
cortex/src/test/java/org/opennms/nephron/cortex/CortexIoPipelineIT.java
Outdated
Show resolved
Hide resolved
NMS-13452: GC global state of CortexIo write transform
swachter
force-pushed
the
jira/NMS-13373
branch
from
4de91e5
to
9d6a709
Compare
Issues:
This PR
cortexConsideredHosts) that allow to specify if / what samples for aggregations for hosts are written.