Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TI MISP] Add datastream for Attributes API endpoint #4136

Merged
merged 29 commits into from Apr 4, 2023
Merged

[TI MISP] Add datastream for Attributes API endpoint #4136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
36e8140
[TI MISP] Add datastream for Attributes API endpoint
legoguy1000 Sep 6, 2022
78f354f
update changelog
legoguy1000 Sep 6, 2022
52247ee
fix test config
legoguy1000 Sep 6, 2022
b444bfd
Merge remote-tracking branch 'elastic/main' into 4115-misp-attributes
andrewkroh Oct 31, 2022
95a34ea
Merge remote-tracking branch 'upstream/main' into 4115-misp-attributes
kcreddy Jan 31, 2023
bf08101
refactor readme
kcreddy Feb 6, 2023
c84350b
add event sub-structure to threat datastream
kcreddy Feb 6, 2023
490c650
Add org id ECS
kcreddy Feb 6, 2023
bcfe1c2
Add tags, decay score fields
kcreddy Feb 6, 2023
5970d13
add event.analysis
kcreddy Feb 7, 2023
568a751
add event fields
kcreddy Feb 7, 2023
b95ae3b
group event and attribute
kcreddy Feb 7, 2023
7331e00
update fingerprint
kcreddy Feb 7, 2023
dd1ff4f
handle decay_score
kcreddy Feb 9, 2023
f935bb9
add object fields
kcreddy Feb 9, 2023
7a2da68
update system test config
kcreddy Feb 9, 2023
bebd75c
null check
kcreddy Feb 9, 2023
b75aea3
update cursor
kcreddy Feb 9, 2023
8550aa7
sample syst test
kcreddy Feb 9, 2023
37b45b6
remove decay_score ref
kcreddy Feb 9, 2023
0c8aed3
Map misp fields to ECS threat
kcreddy Mar 30, 2023
02ebd67
Update README
kcreddy Mar 30, 2023
f4d0b60
Merge remote-tracking branch 'upstream/main' into 4115-misp-attributes
kcreddy Mar 30, 2023
0b1b1e5
Update tests
kcreddy Mar 30, 2023
5705573
Update sample events and read me
kcreddy Mar 30, 2023
c1cd91b
Remove unnecesary conditionls in threat
kcreddy Apr 3, 2023
fcb8908
Address PR review in threat attr
kcreddy Apr 3, 2023
051a0a0
Update misp events as per PR review
kcreddy Apr 3, 2023
e757adb
Address PR review
kcreddy Apr 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 9 additions & 1 deletion packages/ti_misp/_dev/build/docs/README.md
View file
Open in desktop
Expand Up @@ -13,4 +13,12 @@ The filters themselves are based on the [MISP API documentation](https://www.cir

{{fields "threat"}}

{{event "threat"}}
{{event "threat"}}

### Threat Attributes

The MISP integration configuration allows to set the polling interval, how far back it should look initially, and optionally any filters used to filter the results.
This data stream uses the `/attributes/restSearch` API endpoint which returns more granular information regarding MISP attributes and additional information.

{{fields "threat_attributes"}}

162 changes: 162 additions & 0 deletions packages/ti_misp/_dev/deploy/docker/files/config.yml
View file
Open in desktop
Expand Up @@ -314,3 +314,165 @@ rules:
{
"response": []
}
- path: /attributes/restSearch
methods: ["POST"]
request_headers:
Authorization: "test"
Content-Type: application/json
request_body: /^{"limit":"10","page":"1","returnFormat":"json","timestamp":/
responses:
- status_code: 200
body: |-
{
"response": {
"Attribute": [
{
"id": "1",
"event_id": "1",
"object_id": "0",
"object_relation": null,
"category": "External analysis",
"type": "link",
"to_ids": false,
"uuid": "542e4cbd-ee78-4a57-bfb8-1fda950d210b",
"timestamp": "1412320445",
"distribution": "5",
"sharing_group_id": "0",
"comment": "",
"deleted": false,
"disable_correlation": false,
"first_seen": null,
"last_seen": null,
"value": "http://labs.opendns.com/2014/10/02/opendns-and-bash/",
"Event": {
"org_id": "1",
"distribution": "3",
"id": "1",
"info": "OSINT ShellShock scanning IPs from OpenDNS",
"orgc_id": "2",
"uuid": "542e4c9c-cadc-4f8f-bb11-6d13950d210b"
}
},
{
"id": "2",
"event_id": "1",
"object_id": "0",
"object_relation": null,
"category": "External analysis",
"type": "link",
"to_ids": false,
"uuid": "542e4cbe-d560-4e14-9157-1fda950d210b",
"timestamp": "1412320446",
"distribution": "5",
"sharing_group_id": "0",
"comment": "",
"deleted": false,
"disable_correlation": false,
"first_seen": null,
"last_seen": null,
"value": "https://gist.github.com/andrewsmhay/de1cdc63d04c2bbf8c12",
"Event": {
"org_id": "1",
"distribution": "3",
"id": "1",
"info": "OSINT ShellShock scanning IPs from OpenDNS",
"orgc_id": "2",
"uuid": "542e4c9c-cadc-4f8f-bb11-6d13950d210b"
}
},
{
"id": "3",
"event_id": "1",
"object_id": "0",
"object_relation": null,
"category": "External analysis",
"type": "link",
"to_ids": false,
"uuid": "542e4cbe-12a4-4345-b0a4-1fda950d210b",
"timestamp": "1412320446",
"distribution": "5",
"sharing_group_id": "0",
"comment": "",
"deleted": false,
"disable_correlation": false,
"first_seen": null,
"last_seen": null,
"value": "https://gist.githubusercontent.com/andrewsmhay/de1cdc63d04c2bbf8c12/raw/f20402cf5a0c646c63c4521f60587703fe654443/iplist",
"Event": {
"org_id": "1",
"distribution": "3",
"id": "1",
"info": "OSINT ShellShock scanning IPs from OpenDNS",
"orgc_id": "2",
"uuid": "542e4c9c-cadc-4f8f-bb11-6d13950d210b"
}
},
{
"id": "4",
"event_id": "1",
"object_id": "0",
"object_relation": null,
"category": "External analysis",
"type": "text",
"to_ids": false,
"uuid": "542e4ccc-b8fc-44af-959d-6ead950d210b",
"timestamp": "1412320460",
"distribution": "5",
"sharing_group_id": "0",
"comment": "",
"deleted": false,
"disable_correlation": false,
"first_seen": null,
"last_seen": null,
"value": "Shellshock",
"Event": {
"org_id": "1",
"distribution": "3",
"id": "1",
"info": "OSINT ShellShock scanning IPs from OpenDNS",
"orgc_id": "2",
"uuid": "542e4c9c-cadc-4f8f-bb11-6d13950d210b"
}
},
{
"id": "5",
"event_id": "1",
"object_id": "0",
"object_relation": null,
"category": "External analysis",
"type": "comment",
"to_ids": false,
"uuid": "542e4ce7-6120-41c0-8793-e90e950d210b",
"timestamp": "1412320487",
"distribution": "5",
"sharing_group_id": "0",
"comment": "",
"deleted": false,
"disable_correlation": false,
"first_seen": null,
"last_seen": null,
"value": "Data encoded by David André",
"Event": {
"org_id": "1",
"distribution": "3",
"id": "1",
"info": "OSINT ShellShock scanning IPs from OpenDNS",
"orgc_id": "2",
"uuid": "542e4c9c-cadc-4f8f-bb11-6d13950d210b"
}
}
]
}
}
- path: /attributes/restSearch
methods: ["POST"]
request_headers:
Authorization: "test"
Content-Type: application/json
request_body: /^{"limit":"10","page":"2","returnFormat":"json","timestamp":/
responses:
- status_code: 200
body: |-
{
"response": []
}
5 changes: 5 additions & 0 deletions packages/ti_misp/changelog.yml
View file
Open in desktop
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.11.0"
changes:
- description: Add Attributes datastream
type: enhancement
link: https://github.com/elastic/integrations/pull/4136
- version: "1.10.1"
changes:
- description: Drop empty event sets.
Expand Down