Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: new parameter autoupdate_cached_database #2180

Merged
merged 1 commit into from
May 31, 2022

Conversation

sebix
Copy link
Member

@sebix sebix commented May 23, 2022

New parameter autoupdate_cached_database to disable automatic updates (downloads) of cached databases

New parameter `autoupdate_cached_database` to disable automatic updates (downloads) of cached databases
@sebix sebix added feature request Indicates new feature requests component: bots labels May 23, 2022
@sebix sebix added this to the 3.1.0 milestone May 23, 2022
@codecov-commenter
Copy link

codecov-commenter commented May 23, 2022

Codecov Report

Merging #2180 (e4b20d7) into develop (ce60fd4) will increase coverage by 0.00%.
The diff coverage is 33.33%.

@@           Coverage Diff            @@
##           develop    #2180   +/-   ##
========================================
  Coverage    76.34%   76.34%           
========================================
  Files          441      441           
  Lines        23649    23654    +5     
  Branches      3739     3739           
========================================
+ Hits         18055    18059    +4     
  Misses        4857     4857           
- Partials       737      738    +1     
Impacted Files Coverage Δ
intelmq/bots/experts/asn_lookup/expert.py 35.20% <33.33%> (+0.52%) ⬆️
intelmq/bots/experts/domain_suffix/expert.py 43.33% <33.33%> (+0.63%) ⬆️
intelmq/bots/experts/maxmind_geoip/expert.py 21.42% <33.33%> (+0.62%) ⬆️
...telmq/bots/experts/recordedfuture_iprisk/expert.py 42.55% <33.33%> (+0.61%) ⬆️
intelmq/bots/experts/tor_nodes/expert.py 47.05% <33.33%> (+0.63%) ⬆️
intelmq/bots/experts/ripe/expert.py 76.53% <0.00%> (-1.03%) ⬇️

Copy link
Contributor

@waldbauer-certat waldbauer-certat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should maybe consider cronjobs instead of integrating update mechanisms into intelmq bots. but seems fine for me

@sebix
Copy link
Member Author

sebix commented May 31, 2022

We should maybe consider cronjobs instead of integrating update mechanisms into intelmq bots. but seems fine for me

Funny that you suggest that.
cron is a (deprecated) tool to schedule program starts. We use it for exactly this purpose (see https://github.com/certtools/intelmq/tree/develop/contrib/cron-jobs etc.)
And instead of having a set of shell scripts downloading the databases @gethvi migrated the update code to the bots which has major advantages, e.g. the update scripts are aware of the settings of the bots (license keys, database locations) and can also check if the bots are configured. The updates code is also located directly at the same place as the code using it and future improvements could include automated database-updates not relying on cron, but on internal timers (I image that a fully dockerized setup will need that due to the lack of third-party components in the minimalistic container)
See #1524 and https://github.com/certtools/intelmq/blob/version-3.0-ideas/docs/architecture-3.0.md#user-content-general-principles and you can find further reasoning in the CEF-2018-AT-IA-0111 reports and at other places.

@sebix sebix merged commit 79cae29 into certtools:develop May 31, 2022
@sebix sebix deleted the autoupdate_cached_database branch May 31, 2022 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: bots feature request Indicates new feature requests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants