qualys_vmdr: expand documents to map each vulnerability per host #9293
Conversation
efd6
added
enhancement
| "events": body.doc.HOST_LIST_VM_DETECTION_OUTPUT.RESPONSE.HOST_LIST.HOST.map(h, | ||
| h.DETECTION_LIST.DETECTION.map(v, { | ||
| "message": h.with({"DETECTION_LIST": v}).encode_json(), | ||
| }) | ||
| ).flatten(), |
There was a problem hiding this comment.
A less invasive change that does not require any of the ingest pipeline changes, and is not a breaking change, but does not match the issue is
| "events": body.doc.HOST_LIST_VM_DETECTION_OUTPUT.RESPONSE.HOST_LIST.HOST.map(h, | |
| h.DETECTION_LIST.DETECTION.map(v, { | |
| "message": h.with({"DETECTION_LIST": v}).encode_json(), | |
| }) | |
| ).flatten(), | |
| "events": body.doc.HOST_LIST_VM_DETECTION_OUTPUT.RESPONSE.HOST_LIST.HOST.map(h, | |
| h.DETECTION_LIST.DETECTION.map(v, { | |
| "message": h.with({"DETECTION_LIST": {"DETECTION": [v]}}).encode_json(), | |
| }) | |
| ).flatten(), |
This would still map each vuln of each host to a separate document, but would leave the detection in an array of one with the old name. This would mean that it does not break any existing users, but as noted in the issue, "With the current integration, we can't easily create dashboards and enrich vulnerability data (not sure it's possible in an array)." Also the name would now be misleading since it's only an array in shape.
There was a problem hiding this comment.
I don't think there are many people using it in Production at the moment. And I'm not sure we can enrich from the Qualys Knowledge Base from an array (even with 1 value).
I'm in favour to have something cleaner and not store the vulnerability in an array of one value (not sure if it'll be the case or not).
There was a problem hiding this comment.
Thanks. I agree, but wanted to make sure.
🚀 Benchmarks reportTo see the full report comment with |
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
|
fyi @clement-fouque |
| target_field: qualys_vmdr.asset_host_detection.vulnerability | ||
| ignore_missing: true | ||
| - rename: | ||
| if: ctx.qualys_vmdr?.asset_host_detection?.vulnerability != null |
There was a problem hiding this comment.
Does this null check on root field indicate the inner field qualys_vmdr.asset_host_detection.vulnerability.UNIQUE_VULN_ID to be non-null? Just making sure because its same for other processors.
There was a problem hiding this comment.
Good catch. Checking the XSD, they are all minOccurs='0' maxOccurs='1'. Will fix.
| on_failure: | ||
| - append: | ||
| field: error.message | ||
| value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}' |
There was a problem hiding this comment.
I think we should also capture failures for all above date processors to improve telemetry.
💚 Build Succeeded
History
cc @efd6 |
|
81.7% Coverage on New Code
0.0% Duplication on New Code
|
Package qualys_vmdr - 2.0.0 containing this change is available at https://epr.elastic.co/search?package=qualys_vmdr |
Proposed commit message
See title.
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Related issues
Screenshots