Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
278 lines (229 sloc) 8.75 KB

Regenerating a Plugin

The following sections document things you need to know to regenerate the plugin. Regeneration occurs when you want to update the plugin schema to support new actions, triggers, improved title and descriptions, etc. without manually saving your work and updating the JSON schema.

Version

Plugins developed after October 1, 2016 have the ability to be regenerated.

Regenerate

As decribed in the SDK Spec document, plugin skeletons are generated by komand plugin generate python plugin.spec.yaml. If you decide to modify your plugin's scheme, by editing plugin.spec.yaml, you'll need to regenerate the plugin skeleton.

Process:

The following process allows us to pick and choose the changes we would like to keep. This way we can obtain the updated schema as well as select to keep our code.

  1. Commit the current state of your plugin before regeneration. Work will be lost if not committed.
  2. Update the plugin.spec.yaml file with the new schema
  3. Run make regenerate
  4. Use git status and git diff to examine the differences
  5. Use git checkout -- <file> to revert back to a files state before regeneration
  6. Use git add -p to and y|n|s (yes, no, split) to stage changes you want to keep
  7. Commit with results with git commit

Example

Here's an example of plugin regeneration by updating the schema (plugin.spec.yaml) to support:

  • Adding a new action to lookup malware hashes
  • Change plugin title
  • Update plugin version
$ vim plugin.spec.yaml # Updated the schema
$ make regenerate
Generating new plugin skeleton
...
Overlaying plugin skeleton to the current directory

The new skeleton changed everything.

  • Test files are deleted but need to be saved. The new skeleton doens't have them (It starts fresh, removing everything not in it).
  • Version and title name have been included in the new skeleton files
  • The new action gets its own action file

Here's list of the modified files with git status

$ git status
On branch cymon
Your branch is up-to-date with 'origin/cymon'.
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

  modified:   cymon/Makefile
  modified:   cymon/bin/komand_cymon
  modified:   cymon/komand_cymon/actions/__init__.py
  modified:   cymon/komand_cymon/actions/address_blacklist.py
  modified:   cymon/komand_cymon/actions/address_lookup.py
  modified:   cymon/komand_cymon/actions/domain_blacklist.py
  modified:   cymon/komand_cymon/actions/domain_lookup.py
  modified:   cymon/komand_cymon/actions/url_lookup.py
  modified:   cymon/plugin.spec.yaml
  modified:   cymon/setup.py
  deleted:    cymon/tests/address_blacklist.json
  deleted:    cymon/tests/domain_blacklist.json
  deleted:    cymon/tests/found_address_lookup.json
  deleted:    cymon/tests/found_domain_lookup.json
  deleted:    cymon/tests/found_url_lookup.json
  deleted:    cymon/tests/not_found_address_lookup.json
  deleted:    cymon/tests/not_found_domain_lookup.json
  deleted:    cymon/tests/not_found_url_lookup.json

Now, let's get the exact differences of each file with git diff.

$ git diff
...
diff --git a/cymon/plugin.spec.yaml b/cymon/plugin.spec.yaml
index be89427..fddda27 100644
--- a/cymon/plugin.spec.yaml
+++ b/cymon/plugin.spec.yaml
@@ -1,8 +1,8 @@
 plugin_spec_version: v1
 name: cymon
-title: "Cymon"
+title: "Cymon v2 API"
 description: "Cymon Open Threat Intelligence"
-version: 0.1.0
+version: 0.1.1
 vendor: komand

+++ b/cymon/bin/komand_cymon
@@ -3,9 +3,9 @@
 import komand
 from komand_cymon import *

-Name        = 'Cymon'
+Name        = 'Cymon v2 API'
 Vendor      = 'komand'
-Version     = '0.1.0'
+Version     = '0.1.1'
 Description = 'Cymon Open Threat Intelligence'

 class Komand_Cymon(komand.Plugin):
@@ -28,6 +28,8 @@ class Komand_Cymon(komand.Plugin):

         self.add_action(actions.DomainLookup())

+        self.add_action(actions.MalwareLookup())
+
         self.add_action(actions.UrlLookup())


diff --git a/cymon/komand_cymon/actions/__init__.py b/cymon/komand_cymon/actions/__init__.py
index 67b2512..33f800a 100755
--- a/cymon/komand_cymon/actions/__init__.py
+++ b/cymon/komand_cymon/actions/__init__.py
@@ -7,6 +7,8 @@ from domain_blacklist import *

 from domain_lookup import *

+from malware_lookup import *
+
 from url_lookup import *

diff --git a/cymon/setup.py b/cymon/setup.py
index 0b58e13..19ea7bc 100755
--- a/cymon/setup.py
+++ b/cymon/setup.py
@@ -1,7 +1,7 @@
 from setuptools import setup, find_packages

 setup(name='cymon-komand-plugin',
-      version='0.1.0',
+      version='0.1.1',
       description='Cymon Open Threat Intelligence',
       author='komand',
       author_email='',
diff --git a/cymon/tests/address_blacklist.json b/cymon/tests/address_blacklist.json
deleted file mode 100644
index 2c7571d..0000000

     def run(self, params={}):
-        base = 'https://cymon.io:443'
-        url = base + '/api/nexus/v1/domain/%s' % params.get('domain')
-        try:
-          resp = komand.helper.open_url(url)
-          dic = json.loads(resp.read())
-        except:
-          return { 'found': False }
-        dic['found'] = True
-        return dic
+        """TODO: Run action"""
+        return {}
+

diff --git a/cymon/tests/not_found_domain_lookup.json b/cymon/tests/not_found_domain_lookup.json
deleted file mode 100644
index 420ff91..0000000
--- a/cymon/tests/not_found_domain_lookup.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
-  "body": {
-    "action": "domain_lookup",
-    "input": {
-      "domain": ""
-    },
-    "connection": null,
-    "meta": {}
-  },
-  "version": "v1",
-  "type": "action_start"
-}
diff --git a/cymon/tests/not_found_url_lookup.json b/cymon/tests/not_found_url_lookup.json
deleted file mode 100644
index 78d0677..0000000
--- a/cymon/tests/not_found_url_lookup.json
+++ /dev/null
@@ -1,12 +0,0 @@
-{
-  "body": {
-    "action": "url_lookup",
-    "input": {
-      "url": "https://google.com"
-    },
-    "connection": null,
-    "meta": {}
-  },
-  "version": "v1",
-  "type": "action_start"
-}

Next, we'll need to decide what to discard and keep from the new skeleton.

We'll start off by saving the test files that are marked as delete because we still want those. These tests were create for the existing actions we had in this plugin. Since we added a new action and did not remove any of the old ones in our schema change we still the old test files.

The follow command checkouts the previous versions of the files before regenerating marked them as deleted.

git checkout -- tests/*

Next, let's save our existing work in the original action files since the new skeleton cleaned out the code in the run methods.

git checkout -- komand_cymon/actions/address_blacklist.json

Next, we'll update our main python program, keeping the version and title changes we want. We do this by entering y for each change we want to stage. If there was a change we didn't want we would have entered n for no. If there's two changes close together where we want one and not the other we can use s to split them into smaller pieces and then choose to commit the smaller piece.

$ git add -p bin/komand_cymon
diff --git a/cymon/bin/komand_cymon b/cymon/bin/komand_cymon
index b715cd2..a0f3941 100755
--- a/cymon/bin/komand_cymon
+++ b/cymon/bin/komand_cymon
@@ -3,9 +3,9 @@
 import komand
 from komand_cymon import *

-Name        = 'Cymon'
+Name        = 'Cymon v2 API'
 Vendor      = 'komand'
-Version     = '0.1.0'
+Version     = '0.1.1'
 Description = 'Cymon Open Threat Intelligence'

 class Komand_Cymon(komand.Plugin):
Stage this hunk [y,n,q,a,d,/,j,J,g,s,e,?]? y
@@ -28,6 +28,8 @@ class Komand_Cymon(komand.Plugin):

         self.add_action(actions.DomainLookup())

+        self.add_action(actions.MalwareLookup())
+
         self.add_action(actions.UrlLookup())


Stage this hunk [y,n,q,a,d,/,K,g,e,?]? y

The new action file is listed in the Untracked files: section of git status.

...
Untracked files:
  (use "git add <file>..." to include in what will be committed)

  cymon/komand_cymon/actions/malware_lookup.py

We can add it by its path git add komand_cymon/actions/malware_lookup.py.

We repeat this process until the plugin looks the way we want then we commit it e.g. `git commit -m "Regenerating plugin to support new action and updated title and version'.

Video

We have a video documentation this process which may be easier to grasp.

Youtube

You can’t perform that action at this time.