Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Shadowserver Vulnerable SMTP server feed, fix #1984 #2037

Merged
6 commits merged into from
Aug 13, 2021

Conversation

monoidic
Copy link
Contributor

@monoidic monoidic commented Aug 12, 2021

This PR adds support for the Shadowserver Vulnerable SMTP Report and fixes #1984 by differentiating between Vulnerable-HTTP and Accessible-HTTP filenames.

@codecov-commenter
Copy link

codecov-commenter commented Aug 12, 2021

Codecov Report

Merging #2037 (4ab22f7) into develop (7eaf71e) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #2037      +/-   ##
===========================================
- Coverage    75.98%   75.95%   -0.04%     
===========================================
  Files          423      427       +4     
  Lines        22812    22973     +161     
  Branches      3040     3060      +20     
===========================================
+ Hits         17334    17448     +114     
- Misses        4772     4817      +45     
- Partials       706      708       +2     
Impacted Files Coverage Δ
intelmq/bots/parsers/shadowserver/_config.py 97.27% <100.00%> (+0.01%) ⬆️
...lmq/tests/bots/parsers/shadowserver/test_broken.py 100.00% <100.00%> (ø)
.../parsers/shadowserver/test_scan_http_vulnerable.py 100.00% <100.00%> (ø)
.../parsers/shadowserver/test_scan_smtp_vulnerable.py 100.00% <100.00%> (ø)
intelmq/lib/upgrades.py 69.66% <0.00%> (ø)
...lmq/tests/bots/experts/domain_valid/test_expert.py 100.00% <0.00%> (ø)
intelmq/bots/experts/domain_valid/expert.py 42.85% <0.00%> (ø)
intelmq/bots/experts/ripe/expert.py 77.55% <0.00%> (+1.02%) ⬆️

@ghost ghost added this to the 3.0.1 milestone Aug 13, 2021
@ghost ghost added bug Indicates an unexpected problem or unintended behavior component: bots labels Aug 13, 2021
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add an entry in docs/user/bots.rst in the shadowserver feed section? Thanks

otherwise ready for merge

intelmq/bots/parsers/shadowserver/_config.py Outdated Show resolved Hide resolved
@@ -2862,8 +2888,9 @@ def scan_exchange_identifier(field):
('Sinkhole-Events-HTTP-Referer IPv6', 'event6_sinkhole_http_referer', event46_sinkhole_http_referer),
('Spam-URL', 'spam_url', spam_url),
('Vulnerable-ISAKMP', 'scan_isakmp', vulnerable_isakmp),
('Vulnerable-HTTP', 'scan_http', accessible_vulnerable_http),
('Vulnerable-HTTP', 'scan_http_vulnerable', accessible_vulnerable_http),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks!

@monoidic
Copy link
Contributor Author

monoidic commented Aug 13, 2021

The suggested improvement reminded me of #1984, so I also fixed that by making Vulnerable-HTTP match by scan_http_vulnerable rather than scan_http, and added some standalone tests for Vulnerable-HTTP (previously, only Accessible-HTTP had tests).

@monoidic monoidic changed the title Support Shadowserver Vulnerable SMTP server feed Support Shadowserver Vulnerable SMTP server feed, fix #1984 Aug 13, 2021
@ghost
Copy link

ghost commented Aug 13, 2021

The suggested improvement reminded me of #1984,

yeah, me too - but only after you committed the other fix :)

so I also fixed that by making Vulnerable-HTTP match by scan_http_vulnerable rather than scan_http, and added some standalone tests for Vulnerable-HTTP (previously, only Accessible-HTTP had tests).

Thanks! Didn't think that #1984 is based on such a simple mistake.

@ghost ghost merged commit 103a584 into certtools:develop Aug 13, 2021
@ghost
Copy link

ghost commented Aug 13, 2021

Thanks! Also cherry-picked for maintenance in 7a81e37 plus changelog in b76520e

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior component: bots
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Shadowserver Parser can't differentiate between "Accessible HTTP" and "Vulnerable HTTP"
2 participants