3.1.0 Release candidate #2232

sebix announced in Announcements

sebix
Aug 8, 2022
Maintainer

What's Changed

BUG: utils/load_configuration: fix loading JSON with tabs in BUG: utils/load_configuration: fix loading JSON with tabs #2010
CTIP parser: Handle TLP value "unknown" in CTIP parser: Handle TLP value "unknown" #2009
FIX: s/self.parameters/self/ in Shodan collector by @monoidic in FIX: s/self.parameters/self/ in Shodan collector #2020
BUG: provide filename when reading a non-archived file from mail by @kapalex in BUG: provide filename when reading a non-archived file from mail #2021
Add new bot: cut string by delimeter, max length by @mariuskarotkis in Add new bot: cut string by delimeter, max length #1967
Core fixes by @monoidic in Core fixes #2027
DOC: Fix typo by @schacht-certat in DOC: Fix typo #2029
Make update script less verbose by default by @schacht-certat in Make update script less verbose by default #2033
Remove malwardomains parser by @schacht-certat in Remove malwardomains parser #2026
Add new bot: domain valid by @mariuskarotkis in Add new bot: domain valid #1966
Support Shadowserver Vulnerable SMTP server feed, fix Shadowserver Parser can't differentiate between "Accessible HTTP" and "Vulnerable HTTP" #1984 by @monoidic in Support Shadowserver Vulnerable SMTP server feed, fix #1984 #2037
Shadowserver feed updates by @schacht-certat in Shadowserver feed updates #2036
Remove postgresql output bot by @schacht-certat in Remove postgresql output bot #2045
DOC: remove mention of intelmqctl list bot json output by @schacht-certat in DOC: remove mention of intelmqctl list bot json output #2030
Remove workarounds for XMPP bots by @schacht-certat in Remove workarounds for XMPP bots #2048
Use the YAML unsafe loader instead of the safe loader by @schacht-certat in Use the YAML unsafe loader instead of the safe loader #2041
Make the harmonization attribute of the bot private by @schacht-certat in Make the harmonization attribute of the bot private #2049
ENH: imap collector: support unverified connections in ENH: imap collector: support unverified connections #2055
ENH: utils: list_bots: strip descriptions in ENH: utils: list_bots: strip descriptions #2056
lib/harm: minor change to allowed classification.types list in lib/harm: minor change to allowed classification.types list #2057
ENH: small refactor of upgrade functions in ENH: small refactor of upgrade functions #2058
Add Honeypot HTTP Scan feed to Shadowserver parser by @monoidic in Add Honeypot HTTP Scan feed to Shadowserver parser #2047
IDF: Fix and enforce field name restrictions in IDF: Fix and enforce field name restrictions #2059
JinjaExpert - modify message fields using jinja2 by @schacht-certat in JinjaExpert - modify message fields using jinja2 #2040
Implement a clear class hierarchy by @schacht-certat in Implement a clear class hierarchy #2042
DOCS: Docker-compose commands updated by @waldbauer-certat in DOCS: Docker-compose commands updated #2035
Refactoring intelmqctl by @schacht-certat in Refactoring intelmqctl #2067
ENH: lib/bot: fix behavior for unconfigured bots in ENH: lib/bot: fix behavior for unconfigured bots #2054
BUG: fix v300_pipeline_file_removal by @schacht-certat in BUG: fix v300_pipeline_file_removal #2083
Adapt ansible workflow to ansible tests by @schacht-certat in Adapt ansible workflow to ansible tests #2081
BUG: use correct private parameter variable by @schacht-certat in BUG: use correct private parameter variable #2087
DOCS: fix some sphinx warnings by @schacht-certat in DOCS: fix some sphinx warnings #2065
ENH: make json output indented to make it better readable by humans by @schacht-certat in ENH: make json output indented to make it better readable by humans #2079
ENH: use the upstream ansible tasks playbook by @schacht-certat in ENH: use the upstream ansible tasks playbook #2089
Return an INFO string if log file does not yet exist by @schacht-certat in Return an INFO string if log file does not yet exist #2100
MAINT: update the url for the vagrant+ansible test repository by @schacht-certat in MAINT: update the url for the vagrant+ansible test repository #2061
FIX: statistics_host returns if no host value is given by @waldbauer-certat in FIX: statistics_host returns if no host value is given #2104
DOCS: Updated docker installation instructions by @waldbauer-certat in DOCS: Updated docker installation instructions #2069
BUG: fix logrotate ownership options in BUG: fix logrotate ownership options #2111
Add support for new feeds to Dataplane parser by @monoidic in Add support for new feeds to Dataplane parser #2102
Add RPZ file output bot by @mariuskarotkis in Add RPZ file output bot #1962
Add new bot: bro output file by @mariuskarotkis in Add new bot: bro output file #1963
DOC: restructure installation documentation in DOC: restructure installation documentation #2113
ENH: ctip parser: added parameter overwrite in ENH: ctip parser: added parameter overwrite #2112
Add new bot: cut string from string by @mariuskarotkis in Add new bot: cut string from string #1965
ENH: intelmqsetup: revise installation of manager in ENH: intelmqsetup: revise installation of manager #2114
DOC: enhance misp integration document in DOC: enhance misp integration document #2115
ENH: Shodan parser: handle invalid hostnames, move _common_keys out of class, adjust apply_mapping by @monoidic in ENH: Shodan parser: handle invalid hostnames, move _common_keys out of class, adjust apply_mapping #2117
Add JSON parser function to Jinja2 templates by @creideiki in Add JSON parser function to Jinja2 templates #2120
DOC: feeds.yaml: fix HIBP nginx config by @sebix in DOC: feeds.yaml: fix HIBP nginx config #2126
FIX: Fixes update-database script on the last few days of a month. by @gethvi in FIX: Fixes update-database script on the last few days of a month. #2121
DOC: add changelog for PR#2121 by @sebix in DOC: add changelog for PR#2121 #2129
Docs: Remove dead IRC channel by @sebix in Docs: Remove dead IRC channel #2130
Added new shadowserver files to _config.py by @mateodurante in Added new shadowserver files to _config.py #2134
DOC: Update changelog and docs for shadowserver PR#2134 by @sebix in DOC: Update changelog and docs for shadowserver PR#2134 #2135
doc: feeds whishlist: add misp warning lists by @sebix in doc: feeds whishlist: add misp warning lists #2141
docs: feeds whishlist: add avast ioc repo by @sebix in docs: feeds whishlist: add avast ioc repo #2139
ENH: upgrade Python syntax to 3.6+ by @sebix in ENH: upgrade Python syntax to 3.6+ #2136
MS CTIP Azure feed invalid hostname filtering by @monoidic in MS CTIP Azure feed invalid hostname filtering #2144
TST: also test on python 3.10, switch to pytest by @sebix in TST: also test on python 3.10, switch to pytest #2142
FIX: CodeQL exponential backtracking on strings by @waldbauer-certat in FIX: CodeQL exponential backtracking on strings #2148
FIX: Added vscode to gitignore & automated docker build with ACT by @waldbauer-certat in FIX: Added vscode to gitignore & automated docker build with ACT #2153
FIX: threshold expert: configure timeout using Redis cache mixin by @creideiki in FIX: threshold expert: configure timeout using Redis cache mixin #2155
FIX: Fixes Blueliv collector requirements by @gethvi in FIX: Fixes Blueliv collector requirements #2161
Add fix Blueliv collector requirements to CHANGELOG.md by @gutsohnCERT in Add fix Blueliv collector requirements to CHANGELOG.md #2173
FIX: Moved SQLBot to SQLMixin namespace in generic-db-lookup by @waldbauer-certat in FIX: Moved SQLBot to SQLMixin namespace in generic-db-lookup #2146
DOC: installation: remove EOL Fedora versions by @sebix in DOC: installation: remove EOL Fedora versions #2186
BUG: mail collectors: fix IMAP abort handling by @wagner-intevation in BUG: mail collectors: fix IMAP abort handling #2187
bug: fix ctl process manager initialization non-interactive by @wagner-intevation in bug: fix ctl process manager initialization non-interactive #2189
doc: ms azure collector: more precise requirement name by @wagner-intevation in doc: ms azure collector: more precise requirement name #2190
intelmqsetup: fix typo in message by @sebix in intelmqsetup: fix typo in message #2181
gitignore: add coverage file by @sebix in gitignore: add coverage file #2182
ENH: new parameter autoupdate_cached_database by @sebix in ENH: new parameter autoupdate_cached_database #2180
tst: fix reverse_dns test by @wagner-intevation in tst: fix reverse_dns test #2208
pkg: add requests_mock as dev dependency by @sebix in pkg: add requests_mock as dev dependency #2210
intelmqsetup: Fix intelmqsetup accesses version of intelmq_api wrongly leading to crash #2197 by @wagner-intevation in intelmqsetup: Fix #2197 #2198
Fix reconnection in SQL Mixin by @wagner-intevation in Fix reconnection in SQL Mixin #2201
ParserBot: Fix line recovery and message dumping by @wagner-intevation in ParserBot: Fix line recovery and message dumping #2192
FIX: Added check if cymru hostname is an ip address by @waldbauer-certat in FIX: Added check if cymru hostname is an ip address #2215
bug: fix ctip parser handling JSON Payload content by @wagner-intevation in bug: fix ctip parser handling JSON Payload content #2193
docs: workaround for Sphinx 5.1.0 exception: pop from an empty deque sphinx-doc/sphinx#10701 by @wagner-intevation in docs: workaround for sphinx-doc/sphinx#10701 #2225
FIX: Remove AutoShun from IntelMQ as the feed is discontinued by @waldbauer-certat in FIX: Remove AutoShun from IntelMQ as the feed is discontinued #2214
SQL: add MSSQL support and allow storing only a subset of event fields by @creideiki in SQL: add MSSQL support and allow storing only a subset of event fields #2172
ENH: Added personal access token support to GitHub Collector by @waldbauer-certat in ENH: Added personal access token support to GitHub Collector #2145
FIX: Use redis BLMOVE instead of BRPOPLPUSH (deprecated) by @waldbauer-certat in FIX: Use redis BLMOVE instead of BRPOPLPUSH (deprecated) #2149
postgresql output: escape all null bytes by @wagner-intevation in postgresql output: escape all null bytes #2223
Remove malc0de feed due to offline by @gutsohnCERT in Remove malc0de feed due to offline #2184
TST: Fixed regex to hold gcc version by @waldbauer-certat in TST: Fixed regex to hold gcc version #2216
Shadowserver 202207 by @elsif2 in Shadowserver 202207 #2227
FIX: Possible race condition in file collector by @waldbauer-certat in FIX: Possible race condition in file collector #2147
tst: fix threshold expert redis host by @wagner-intevation in tst: fix threshold expert redis host #2209
3.1.0 Release candidate 1 by @sebix in 3.1.0 Release candidate 1 #2230

New Contributors

@mateodurante made their first contribution in Added new shadowserver files to _config.py #2134
@gutsohnCERT made their first contribution in Add fix Blueliv collector requirements to CHANGELOG.md #2173
@wagner-intevation made their first contribution in BUG: mail collectors: fix IMAP abort handling #2187
@elsif2 made their first contribution in Shadowserver 202207 #2227

Full Changelog: 3.0.2...3.1.0-rc1

This discussion was created from the release 3.1.0 Release candidate.

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment