-
Notifications
You must be signed in to change notification settings - Fork 387
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Qualys VMDR] Initial release for the Qualys VMDR #6872
Conversation
🌐 Coverage report
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Initial review: CEL LGTM
Do we have a test where there is only a single element in the response list?
|
Why do you think this? It should be making an array of a single element and that can be mapped over. If this is not the case this is not safe. |
packages/qualys_vmdr/data_stream/asset_host_detection/agent/stream/input.yml.hbs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do Qualys provide better docs?
packages/qualys_vmdr/data_stream/asset_host_detection/manifest.yml
Outdated
Show resolved
Hide resolved
packages/qualys_vmdr/data_stream/asset_host_detection/manifest.yml
Outdated
Show resolved
Hide resolved
"dataset": "qualys_vmdr.asset_host_detection", | ||
"ingested": "2023-07-12T07:41:28Z", | ||
"kind": "alert", | ||
"original": "{\"DETECTION_LIST\":{\"DETECTION\":[{\"FIRST_FOUND_DATETIME\":\"2023-06-28T06:04:26Z\",\"IS_DISABLED\":\"0\",\"IS_IGNORED\":\"0\",\"LAST_FOUND_DATETIME\":\"2023-07-03T06:23:47Z\",\"LAST_PROCESSED_DATETIME\":\"2023-07-03T06:25:17Z\",\"LAST_TEST_DATETIME\":\"2023-07-03T06:23:47Z\",\"LAST_UPDATE_DATETIME\":\"2023-07-03T06:25:17Z\",\"QID\":\"91681\",\"RESULTS\":\"\",\"SEVERITY\":\"5\",\"SSL\":\"0\",\"STATUS\":\"Active\",\"TIMES_FOUND\":\"11\",\"TYPE\":\"Confirmed\"}]},\"DNS\":\"\",\"DNS_DATA\":{\"DOMAIN\":\"\",\"FQDN\":\"\",\"HOSTNAME\":\"\"},\"ID\":\"12048633\",\"IP\":\"10.50.2.111\",\"LAST_PC_SCANNED_DATE\":\"2023-06-28T09:58:12Z\",\"LAST_SCAN_DATETIME\":\"2023-07-03T06:25:17Z\",\"LAST_VM_SCANNED_DATE\":\"2023-07-03T06:23:47Z\",\"LAST_VM_SCANNED_DURATION\":\"1113\",\"NETBIOS\":\"\",\"OS\":\"\",\"TRACKING_METHOD\":\"IP\"}", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Design question. Do we want to retain the original XML here rather than the JSON? We can do this, though it obviously adds network cost and a small amount of complexity to the CEL code/ingest preamble.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we do not have any processor for ingest pipeline which will decode the XML. So, we have directly ingested json
format inside pipeline test. Since sample_event.json
takes event.original
directly from ingest pipeline, it sets it in the json
format.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is where the additional complexity comes from. The body send to ingest would not just be the JSON, it would be {"json": <JSON message>, "xml": <original XML>}
. Entirely doable, but I wondering if it is worth doing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it could be considered a bit different.
- If we want to have event.original, it needs to be the original body before any sort of decode.
- If that is not plausible, we should not have the option to store event.original, as it would go against what the field stands for (also for compliance reasons). In certain cases we have rather added support for this in the input itself (http_endpoint is one example).
If we feel that there is added complexity, we can go ahead and implement the initial version without event.original, while we discuss the best approach to add it in later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@P1llus, currently we do have event.orignal
but we are considering JSON object (creates immediate after decoding actual XML). Is that okay? because if we remove event.original
then preserve original event functionality will not work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@efd6 as Marius is on PTO, could you review Piyush's comment here? It's the last outstanding point, so if we're ok with this approach we can merge the PR now vs waiting for Marius to return.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Marius' view was that we could add the event.original
handling later. I'm OK with this too.
packages/qualys_vmdr/data_stream/knowledge_base/agent/stream/input.yml.hbs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Still LGTM
Package qualys_vmdr - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=qualys_vmdr |
@SpencerLN @clement-fouque Qualys integration now available if you'd like to test and provide any feedback. |
What does this PR do?
Integration release checklist
This checklist is intended for integrations maintainers to ensure consistency
when creating or updating a Package, Module or Dataset for an Integration.
All changes
New Package
Dashboards changes
Log dataset changes
How to test this PR locally
Screenshots
Automated Test