Permalink
Browse files

Merge branch 'maintenance' into develop

  • Loading branch information...
wagner-certat committed Jan 15, 2019
2 parents 7c6d015 + 0ddec51 commit e10c60334cbb87451bd9565381f9f568ba1f99c1
@@ -67,7 +67,38 @@ CHANGELOG
### Known issues


1.1.1 (unreleased)
1.1.2 (unreleased)
------------------

### Core

### Development

### Harmonization

### Bots
#### Collectors

#### Parsers

#### Experts

#### Outputs

### Documentation

### Packaging

### Tests

### Tools

### Contrib

### Known issues


1.1.1 (2019-01-15)
------------------

### Core
@@ -94,6 +125,39 @@ CHANGELOG
### Core

### Harmonization
Update to 2018-09-26 version. New values are per taxonomy:
- Taxonomy 'intrusions':
- "application-compromise"
- "burglary"
- "privileged-account-compromise"
- "unprivileged-account-compromise"
- Taxonomy 'fraud':
- "copyright"
- "masquerade"
- "unauthorized-use-of-resources"
- Taxonomy 'information content security':
- "data-loss"
- Taxonomy 'vulnerable':
- "ddos-amplifier"
- "information-disclosure"
- "potentially-unwanted-accessible"
- "vulnerable-system"
- "weak-crypto"
- Taxonomy 'availability':
- "dos"
- "outage"
- "sabotage"
- Taxonomy 'abusive-content':
- "harmful-speech"
- "violence"
- Taxonomy 'malicious code':
- "malware-distribution"
- Taxonomy 'information-gathering':
- "social-engineering"
- "sniffing"
- Taxonomy 'information content security':
- "Unauthorised-information-access"
- "Unauthorised-information-modification"

### Bots
#### Collectors
@@ -146,6 +210,7 @@ CHANGELOG
- Add text and more context to error messages.
- README: Fix 'modify' to 'update' (#1340).
- Handle empty rules file (#1343).
- `intelmq.bots.experts.idea.expert`: Add mappings for new harmonization `classification.type` values, see above.

#### Outputs
- `intelmq.bots.outputs.redis`:
@@ -190,6 +255,9 @@ CHANGELOG
- Handle collector's `feed.name` and `feed.provider` (#1314).

### Known issues
- Bots started with IntelMQ-Manager stop when the webserver is restarted (#952).
- Tests: capture logging with context manager (#1342).
- stomp collector bot constantly uses 100% of CPU (#1364).


1.1.0 (2018-09-05)
10 NEWS.md
@@ -13,13 +13,19 @@ See the changelog for a full list of changes.
### Configuration
The bot `intelmq.bots.experts.ripencc_abuse_contact.expert` has been renamed to `intelmq.bots.experts.ripe.expert`, the compatibility shim will be removed in version 3.0. Adapt your `runtime.conf` accordingly.


1.1.2 Bugfix release (unreleased)
---------------------------------

### Configuration

### Libraries

### Postgres databases


1.1.1 Bugfix release (unreleased)
----------------------------------
1.1.1 Bugfix release (2019-01-15)
---------------------------------

### Tools

@@ -10,6 +10,18 @@ intelmq (1.2.0~alpha1-1) unstable; urgency=medium

-- Sebastian Wagner <wagner@cert.at> Sat, 07 Jul 2018 11:43:23 +0200

intelmq (1.1.2~alpha1-1) unstable; urgency=medium

* Update to version 1.1.2.

-- Wagner Sebastian <wagner@cert.at> Tue, 15 Jan 2019 17:03:15 +0100

intelmq (1.1.1-1) stable; urgency=medium

* Update to version 1.1.1.

-- Wagner Sebastian <wagner@cert.at> Tue, 15 Jan 2019 16:11:29 +0100

intelmq (1.1.1~alpha1-1) unstable; urgency=medium

* Update to version 1.1.1 alpha 1
@@ -123,36 +123,66 @@ Sanitation accepts string 'true' and 'false' and integers 0 and 1.

### ClassificationType

classification.type type. Allowed values are:
`classification.type` type.

The mapping follows
Reference Security Incident Taxonomy Working Group – RSIT WG
https://github.com/enisaeu/Reference-Security-Incident-Taxonomy-Task-Force/
with extensions.

Allowed values are:
* application-compromise
* backdoor
* blacklist
* botnet drone
* brute-force
* burglary
* c&c
* compromised
* copyright
* data-loss
* ddos
* ddos-amplifier
* defacement
* dga domain
* dos
* dropzone
* exploit
* harmful-speech
* ids alert
* infected system
* information-disclosure
* leak
* malware
* malware configuration
* malware-distribution
* masquerade
* other
* outage
* phishing
* potentially-unwanted-accessible
* privileged-account-compromise
* proxy
* ransomware
* sabotage
* scanner
* sniffing
* social-engineering
* spam
* test
* tor
* unauthorized-login
* Unauthorised-information-access
* Unauthorised-information-modification
* unauthorized-command
* unauthorized-login
* unauthorized-use-of-resources
* unknown
* unprivileged-account-compromise
* violence
* vulnerable client
* vulnerable service
* vulnerable-system
* weak-crypto

### DateTime

@@ -501,7 +501,7 @@ Bots configured as `continuous` will never exit except if there is an error and
#### Reloading
Whilts restart is a mere stop & start, performing `intelmqctl reload <bot_id>` will not stop the bot, permitting it to keep the state: the same common behavior as for (linux) daemons. It will initialize again (including reading all configuration again) after the current action is finished. Also, the rate limit/sleep is continued (with the *new* time) and not interrupted like with the restart command. So if you have a collector with a rate limit of 24 h, the reload does not trigger a new fetching of the source at the time of the reload, but just 24 h after the last run – with the new configuration.
Whilst restart is a mere stop & start, performing `intelmqctl reload <bot_id>` will not stop the bot, permitting it to keep the state: the same common behavior as for (Linux) daemons. It will initialize again (including reading all configuration again) after the current action is finished. Also, the rate limit/sleep is continued (with the *new* time) and not interrupted like with the restart command. So if you have a collector with a rate limit of 24 h, the reload does not trigger a new fetching of the source at the time of the reload, but just 24 h after the last run – with the new configuration.
Which state the bots are keeping depends on the bots of course.
#### Forcing reset pipeline and cache (be careful)
@@ -1,4 +1,7 @@
# -*- coding: utf-8 -*-
"""
IDEA classification: https://idea.cesnet.cz/en/classifications
"""
from collections import Sequence, Mapping
from base64 import b64decode
from uuid import uuid4
@@ -51,6 +54,29 @@ class IdeaExpertBot(Bot):
"test": "Test",
"unauthorized-command": "Intrusion.AdminCompromise",
"unauthorized-login": "Intrusion.AdminCompromise",
"violence": "Abusive.Violence",
"data-loss": "Information",
"burglary": "Intrusion",
"weak-crypto": "Vulnerable.Config",
"Unauthorised-information-access": "Information.UnauthorizedAccess",
"privileged-account-compromise": "Intrusion.AdminCompromise",
"potentially-unwanted-accessible": "Vulnerable.Open",
"application-compromise": "Intrusion.AppCompromise",
"unauthorized-use-of-resources": "Fraud.UnauthorizedUsage",
"masquerade": "Fraud.Scam",
"harmful-speech": "Abusive.Harassment",
"unprivileged-account-compromise": "Intrusion.UserCompromise",
"social-engineering": "Recon.SocialEngineering",
"dos": "Availability.DoS",
"information-disclosure": "Information.UnauthorizedAccess",
"sniffing": "Recon.Sniffing",
"vulnerable-system": "Vulnerable.Config",
"Unauthorised-information-modification": "Information.UnauthorizedModification",
"sabotage": "Availability.Sabotage",
"malware-distribution": "Malware",
"outage": "Availability.Outage",
"ddos-amplifier": "Intrusion.Botnet",
"copyright": "Fraud.Copyright",
}

type_to_source_type = {
@@ -63,6 +89,7 @@ class IdeaExpertBot(Bot):
"dga domain": "DGA",
"proxy": "Proxy",
"tor": "Tor",
"malware-distribution": "Malware"
}

def init(self):
@@ -1,40 +1,70 @@
# -*- coding: utf-8 -*-
"""
The mapping follows
Reference Security Incident Taxonomy Working Group – RSIT WG
https://github.com/enisaeu/Reference-Security-Incident-Taxonomy-Task-Force/
with extensions.
"""

from intelmq.lib.bot import Bot

# FIXME: this dict should be on a separated file

TAXONOMY = {
# type # taxonomy
"phishing": "fraud",
"proxy": "Other",
"ddos": "availability",
# sorted!
"spam": "abusive content",
"harmful-speech": "abusive-content",
"violence": "abusive-content",
"ddos": "availability",
"dos": "availability",
"outage": "availability",
"sabotage": "availability",
"copyright": "fraud",
"masquerade": "fraud",
"phishing": "fraud",
"unauthorized-use-of-resources": "fraud",
"Unauthorised-information-access": "information content security",
"Unauthorised-information-modification": "information content security",
"data-loss": "information content security",
"dropzone": "information content security", # not in ENISA eCSIRT-II taxonomy
"leak": "information content security", # not in ENISA eCSIRT-II taxonomy
"scanner": "information gathering",
"dropzone": "information content security",
"malware": "malicious code",
"sniffing": "information-gathering",
"social-engineering": "information-gathering",
"brute-force": "intrusion attempts",
"exploit": "intrusion attempts",
"ids alert": "intrusion attempts", # ENISA eCSIRT-II taxonomy: 'ids-alert'
"application-compromise": "intrusions",
"backdoor": "intrusions", # not in ENISA eCSIRT-II taxonomy
"burglary": "intrusions",
"compromised": "intrusions", # not in ENISA eCSIRT-II taxonomy,
"defacement": "intrusions", # not in ENISA eCSIRT-II taxonomy
"privileged-account-compromise": "intrusions",
"unauthorized-command": "intrusions", # not in ENISA eCSIRT-II taxonomy
"unauthorized-login": "intrusions", # not in ENISA eCSIRT-II taxonomy
"unprivileged-account-compromise": "intrusions",
"botnet drone": "malicious code",
"ransomware": "malicious code",
"c&c": "malicious code", # ENISA eCSIRT-II taxonomy: 'c2server'
"dga domain": "malicious code",
"malware configuration": "malicious code",
"c&c": "malicious code",
"exploit": "intrusion attempts",
"brute-force": "intrusion attempts",
"ids alert": "intrusion attempts",
"defacement": "intrusions",
"compromised": "intrusions",
"backdoor": "intrusions",
"vulnerable service": "vulnerable",
"vulnerable client": "vulnerable",
"infected system": "malicious code", # ENISA eCSIRT-II taxonomy: 'infected-system'
"malware": "malicious code",
"malware configuration": "malicious code", # ENISA eCSIRT-II taxonomy: 'malware-configuration'
"malware-distribution": "malicious code",
"ransomware": "malicious code",
"blacklist": "other",
"unknown": "other",
"test": "test",
"other": "other",
"proxy": "other",
"tor": "other",
"leak": "information content security",
"infected system": "malicious code",
'unauthorized-login': 'intrusions',
'unauthorized-command': 'intrusions',
"unknown": "other",
"test": "test",
"ddos-amplifier": "vulnerable",
"information-disclosure": "vulnerable",
"potentially-unwanted-accessible": "vulnerable",
"vulnerable client": "vulnerable", # not in ENISA eCSIRT-II taxonomy
"vulnerable service": "vulnerable", # not in ENISA eCSIRT-II taxonomy
"vulnerable-system": "vulnerable",
"weak-crypto": "vulnerable",
}


Oops, something went wrong.

0 comments on commit e10c603

Please sign in to comment.