Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new bot: domain valid #1966

Merged
14 commits merged into from
Aug 13, 2021
Merged

Conversation

mariuskarotkis
Copy link
Contributor

Add new bot: domain valid

@ghost
Copy link

ghost commented Jun 7, 2021

Hi @mariuskarotkis and thanks for the contribution

On the first glance, this looks very similar to the domain suffix expert, which uses the PSL. Is there a specific reason you did create a new specialized bot instead of utilizing the domain-suffix expert and filtering on the source.domain_suffix field?

@mariuskarotkis
Copy link
Contributor Author

mariuskarotkis commented Jun 7, 2021

Hi @wagner-certat
This expert allows only valid domains.

I didn't know about the expert domain-suffix.
I'll try it, thanks.

@codecov-commenter
Copy link

codecov-commenter commented Aug 6, 2021

Codecov Report

Merging #1966 (b2acff3) into develop (7eaf71e) will decrease coverage by 0.07%.
The diff coverage is 60.65%.

@@             Coverage Diff             @@
##           develop    #1966      +/-   ##
===========================================
- Coverage    75.98%   75.90%   -0.08%     
===========================================
  Files          423      425       +2     
  Lines        22812    22934     +122     
  Branches      3040     3058      +18     
===========================================
+ Hits         17334    17409      +75     
- Misses        4772     4817      +45     
- Partials       706      708       +2     
Impacted Files Coverage Δ
intelmq/bots/experts/domain_valid/expert.py 42.85% <42.85%> (ø)
...lmq/tests/bots/experts/domain_valid/test_expert.py 100.00% <100.00%> (ø)
intelmq/bots/experts/ripe/expert.py 77.55% <0.00%> (+1.02%) ⬆️

docs/user/bots.rst Outdated Show resolved Hide resolved
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a review of the code and I have a few suggestions/required changes.

docs/user/bots.rst Outdated Show resolved Hide resolved
docs/user/bots.rst Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
docs/user/bots.rst Outdated Show resolved Hide resolved
docs/user/bots.rst Outdated Show resolved Hide resolved
docs/user/bots.rst Outdated Show resolved Hide resolved
@ghost ghost added this to the 3.1.0 milestone Aug 13, 2021
@ghost ghost added component: bots feature Indicates new feature requests or new features labels Aug 13, 2021
@ghost ghost self-assigned this Aug 13, 2021
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a file intelmq/tests/bots/experts/domain_valid/tlds-alpha-by-domain.txt.license with the following content:

SPDX-FileCopyrightText: 2021 Marius Karotkis <marius.karotkis@gmail.com>
SPDX-License-Identifier: AGPL-3.0-or-later

With these proposals incorporated, the bot runs fine and the checks should succeed. Also tested locally.

If you add the suggestions, I'll finally merge the PR after a long odyssey :)

intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
intelmq/bots/experts/domain_valid/expert.py Outdated Show resolved Hide resolved
@ghost ghost merged commit d7af53c into certtools:develop Aug 13, 2021
waldbauer-certat pushed a commit that referenced this pull request Aug 16, 2021
* Add new bot: domain valid

* Add license, add description

* Add requirements

* Update domain valid bot description, tests

* Update domain valid bot description, reduce data file

* Add cron and update TLD file function

* fix

* fix

* fix

* fix

* fix

* fix

* Update after develop branch merge

Co-authored-by: Wagner <wagner@cert.at>
waldbauer-certat pushed a commit that referenced this pull request Sep 9, 2021
* Add new bot: domain valid

* Add license, add description

* Add requirements

* Update domain valid bot description, tests

* Update domain valid bot description, reduce data file

* Add cron and update TLD file function

* fix

* fix

* fix

* fix

* fix

* fix

* Update after develop branch merge

Co-authored-by: Wagner <wagner@cert.at>
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: bots feature Indicates new feature requests or new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants