Skip to content
Permalink
Browse files

ENH: New bot CSV Converter

  • Loading branch information...
wagner-certat committed Nov 4, 2019
1 parent b6a46e5 commit bc95ecdd13f421c0dcdcfbd120890394fc5d83b2
@@ -154,6 +154,7 @@ CHANGELOG
- Add geohash expert.
- `intelmq.bot.experts.generic_db_lookup.expert`
- new optional parameter `engine` with `postgresql` (default) and `sqlite` (new) as possible values.
- `intelmq.bots.experts.csv_converter`: Added as converter to CSV.

#### Outputs
- Add `intelmq.bots.outputs.touch.output`.
@@ -39,6 +39,7 @@
- [Experts](#experts)
- [Abusix](#abusix)
- [ASN Lookup](#asn-lookup)
- [CSV Converter](#csv-converter)
- [Copy Extra](#copy-extra)
- [Cymru Whois](#cymru-whois)
- [Deduplicator](#deduplicator)
@@ -1406,6 +1407,28 @@ Note: the '<' '>' characters only are syntactic markings, no shell redirection i

* * *

### CSV Converter


#### Information:
* `name`: `intelmq.bots.experts.csv_converter.expert
* `lookup`: no
* `public`: yes
* `cache (redis db)`: none
* `description`: Converts an event to CSV format, saved in the `output` field.
#### Configuration Parameters:
* `delimiter`: String, default `","`
* `fieldnames`: Comma-separated list of field names, e.g. `"time.source,classification.type,source.ip"`

#### Usage

To use the CSV-converted data in an output bot - for example in a file output,
use the configuration parameter `single_key` of the output bot and set it to `output`.

* * *

### Copy Extra

#### Information:
@@ -683,6 +683,14 @@
"database": "/opt/intelmq/var/lib/bots/asn_lookup/ipasn.dat"
}
},
"CSV Converter": {
"description": "Converts data to CSV.",
"module": "intelmq.bots.experts.csv_converter.expert",
"parameters": {
"delimiter": ",",
"fieldnames": "time.source,classification.type,source.ip"
}
},
"Abusix": {
"description": "Abusix is the bot responsible to get the correspondent abuse contact from source IP and destination IP of the events.",
"module": "intelmq.bots.experts.abusix.expert",
No changes.
@@ -0,0 +1,28 @@
# -*- coding: utf-8 -*-
import csv
import io
from intelmq.lib.bot import Bot


class CSVConverterExpertBot(Bot):

def init(self):
self.fieldnames = self.parameters.fieldnames.split(',')
self.delimiter = getattr(self.parameters, 'delimiter', ',')

def process(self):
event = self.receive_message()
event.set_default_value('')
out = io.StringIO()
writer = csv.writer(out, delimiter=self.delimiter)
row = []
for field in self.fieldnames:
row.append(event[field])
writer.writerow(row)
event['output'] = out.getvalue().rstrip()

self.send_message(event)
self.acknowledge_message()


BOT = CSVConverterExpertBot

0 comments on commit bc95ecd

Please sign in to comment.
You can’t perform that action at this time.