Add ML modules to Apache integration readme #2640
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
Agreed that automating this is the best approach. @mtojek what do you think about automatically adding this content to the docs for integrations that have ML jobs? |
There was a problem hiding this comment.
Change LGTM, but I'd like to know first if we need to bump up the Kibana version constraint in manifest.yml.
For example, if ML jobs are present since 7.16.3, then we have to bump up the constraint to:
kibana.version: "^7.16.3 || ^8.0.0" (follows semver rules).
| @@ -32,3 +32,21 @@ generated by the `mod_status` module. | |||
| {{event "status"}} | |||
|
|
|||
| {{fields "status"}} | |||
|
|
|||
| ## ML Modules | |||
There was a problem hiding this comment.
@peteharverson Do you know since which Kibana are these jobs present? We might need to bump up Kibana's constraint in this PR's manifest.
There was a problem hiding this comment.
@mtojek support for ML modules was added to Fleet for 7.13.0 - elastic/kibana#94950, and the package itself was updated shortly after that PR - #910. So the Kibana constraint looks ok to me.
| | visitor_rate_apache | HTTP Access Logs: Detect unusual visitor rates | | ||
| | status_code_rate_apache | HTTP Access Logs: Detect unusual status code rates | | ||
| | source_ip_url_count_apache | HTTP Access Logs: Detect unusual source IPs - high distinct count of URLs | | ||
| | source_ip_request_rate_apache | HTTP Access Logs: Detect unusual source IPs - high request rates | | ||
| | low_request_rate_apache | HTTP Access Logs: Detect low request rates | |
There was a problem hiding this comment.
@peteharverson Is there any way/list we can grab these fields or mark them in packages? This would be the first step to automate things.
There was a problem hiding this comment.
@mtojek the module JSON file holding the ML anomaly detection job definitions is in kibana/ml_module. The jobs property inside attributes is an array of job objects. The format used here for each job would be
job.id | job.config.description
Currently we have published ML modules for the apache and nginx packages, and there are two more packages being worked on currently - problem child - #2115, and dga - #2352.
|
@bmorelli25 Absolutely, but let's first figure out where are these fields defined and how we can grab them. Pinging @peteharverson. |
What does this PR do?
Adds list of machine learning jobs to the Apache integration readme.
The ML module is listed as an asset when you view the integration in Kibana, for example:
... However there is currently no further information about what that module provides.
Ideally the information in the table could be automatically included, since it's all just strings from https://github.com/elastic/integrations/blob/main/packages/apache/kibana/ml_module/apache-Logs-ml.json
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Related issues
Screenshots