Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redo threat intel plugin migration with new content packs #107

Merged
merged 7 commits into from
Oct 11, 2018

Conversation

kmerz
Copy link
Member

@kmerz kmerz commented Sep 6, 2018

The threat intel plugin used the content pack mechanism to install dependency configuration.
For that it read the configuration via a content pack. Since we rewrote the content pack code,
we need to adjust the migration as well.

  • We split up the on content pack into 5 smaller ones which are independent from one another
  • We do not install the content pack right away, but let the user decide what he wants to use as a lookuptable. We only add the content pack into the database.
  • We use the new content pack mechanism.

@kmerz kmerz added this to the 3.0.0 milestone Sep 6, 2018
@kmerz kmerz self-assigned this Sep 6, 2018
@kmerz kmerz changed the title Add new content-packs for the migration Redo threat intel plugin migration with new content packs Sep 7, 2018
@kmerz kmerz requested a review from bernd September 7, 2018 13:07
@AutoValue
@WithBeanGetter
public static abstract class MigrationCompleted {
@JsonProperty("content_bundle_id")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename this to content_pack_ids.

public abstract List<String> contentBundleIds();

@JsonCreator
public static MigrationCompleted create(@JsonProperty("content_bundle_id") final List<String> contentBundleIds) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

@bernd bernd self-assigned this Oct 10, 2018
Copy link
Member

@bernd bernd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my comment about the JSON files. Otherwise it looks good and works for me. 👍

@@ -0,0 +1 @@
{"id":"9350a70a-8453-f516-7041-517b4df0b832","rev":1,"v":"1","name":"Tor Exit Node List - Threat Intel Plugin","summary":"The Tor Exit Node List Lookup Table of the Threat Intel Plugin","description":"This content pack is part of the [Graylog Threat Intel Plugin](http://github.com/graylog-labs/graylog-plugin-threatintel).\n\n*Please do not delete it manually if you consider to use the Threat Intel Plugin's functions. It contains important resources required by the plugin.*\n\nThis is the lookup table for the TOR (The Onion Router) Exit Node List, listing Exit Nodes of the TOR Network . This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually.","vendor":"Graylog <hello@graylog.com>","url":"https://github.com/kmerz/noodle","requires":[{"type":"plugin-version","plugin":"org.graylog.plugins.threatintel.ThreatIntelPlugin","version":">=3.0.0-alpha.2"},{"type":"server-version","version":">=3.0.0-alpha.2+af8d8e0"}],"parameters":[],"entities":[{"id":"5ac762873d274666e34eca87","type":{"name":"lookup_adapter","version":"1"},"v":"1","data":{"name":{"type":"string","value":"tor-exit-node"},"title":{"type":"string","value":"Tor Exit Node"},"description":{"type":"string","value":"This is the data adapter for the TOR (The Onion Router) Exit Node List, listing Exit Nodes of the TOR Network . This adapter is used internally by Graylog's Threat Intel Plugin. Do not delete it manually."},"configuration":{"type":{"type":"string","value":"torexitnode"}}}},{"id":"5ac762873d274666e34eca80","type":{"name":"lookup_cache","version":"1"},"v":"1","data":{"name":{"type":"string","value":"threat-intel-uncached-adapters"},"title":{"type":"string","value":"Threat Intel Uncached Adapters"},"description":{"type":"string","value":"Certain downloaded lists do not require caching because they are in memory already and use this no-op cache."},"configuration":{"type":{"type":"string","value":"none"}}}},{"id":"5ac762873d274666e34eca8c","type":{"name":"lookup_table","version":"1"},"v":"1","data":{"name":{"type":"string","value":"tor-exit-node-list"},"title":{"type":"string","value":"Tor Exit Node List"},"description":{"type":"string","value":"This is the lookup table for the TOR (The Onion Router) Exit Node List, listing Exit Nodes of the TOR Network . This lookup table is used internally by Graylog's Threat Intel Plugin. Do not delete it manually."},"cache_name":{"type":"string","value":"5ac762873d274666e34eca80"},"data_adapter_name":{"type":"string","value":"5ac762873d274666e34eca87"},"default_single_value":{"type":"string","value":""},"default_single_value_type":{"type":"string","value":"NULL"},"default_multi_value":{"type":"string","value":""},"default_multi_value_type":{"type":"string","value":"NULL"}}}]}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why, but the url of this plugin is set to "github.com/kmerz/noodle". 😄

Also, please prettify all the JSON files. That makes it easier to read them. 😃

Copy link
Member

@bernd bernd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@bernd bernd merged commit 1bd6338 into master Oct 11, 2018
@bernd bernd deleted the content-pack-migration branch October 11, 2018 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants