Unable to install custom misp-module created

[davehouser1]

Goal:

• Create a custom expansion / enrichment misp-module, have it integrate into the web interface, able to enable and use.

Note: I am an armature python developer, I am also very new to MISP.

Problem:

• New misp-module does not appear on the web interface to enable.
• Also does not appear in list of modules when querying the API

Work environment

Questions	Answers
Type of issue	Support
OS version (server)	Ubuntu 18.04
OS version (client)	Ubuntu 18.04
MISP version / git hash	2.4.129, (Don't know hash of commit, how can I find? I used the installer)
Browser	Firefox 79.0 64-bit

Support Questions

• How can I get a custom misp-module to install?

Logs, screenshots, configuration dump, ...

• Here is my test script, it literally just recognizes a ip.src and prints out a message.

import json
import dns.resolver

misperrors = {'error' : 'Error'}
mispattributes = {'input': ['ip-src'] , 'output' : ['text']}
moduleinfo = {'version': '0.1', 'author': 'dh016d', 'description': 'Test module built by dave', 'module-type': ['expansion', 'hover']}

def handler(q=False):
    if q is False:
        return False
    request = json.loads(q)

    ip = request['ip-src']
    output = []
    output.append(ip)

    if output:
        output = "Hello, this is a test, input shows the ip is " + ip + "!"
        misp_output = {'results': [{'types': mispattributes['output'], 'values': output}]}
    return misp_output

def introspection():
    return mispattributes

def version():
    return moduleinfo

• I put my .py script into /usr/local/src/misp-modules/misp_modules/modules/expansion
• I have tried running the /usr/local/src/misp-modules/tools/update_misp_modules.sh
  • This keeps failing with the error

error: cannot open .git/FETCH_HEAD: Permission denied
ERROR: Command errored out with exit status 255: git fetch -q --tags Check the logs for full command output.

• Tried rebooting

• Tried running as sudo

• Tried redeploying and reinstalling from scratch. This feels like a bad configuration setting I have set up, however I have followed the install instructions

• I have tried checking to see if MISP recognizes modules automatically, it does not seem to.

• Sources I have read and tried to follow already:

  • Module creation: https://github.com/MISP/misp-modules
  • Module creation: https://medium.com/ouspg/quick-guide-to-writing-misp-modules-c53c7bb00914
  • Module creation: https://www.circl.lu/assets/files/misp-training/switch2016/2-misp-modules.pdf
  • Module creation: https://www.misp-project.org/misp-training/3.1-misp-modules.pdf
  • Install: https://www.inspark.nl/misp-threat-intelligence-azure-sentinel/
  • Install: https://misp.github.io/MISP/INSTALL.ubuntu1804/

What am I doing wrong here?

EDIT:
I was able to get the MISP modules loaded on one of the instances of MISP I deployed. The trick was in this link:
#5613

Why is this not in documentation? You need to edit the systemd unit ExecStart to remove the "-s". Seemed to make it work. Is this expected?

However the other git problem still remains when running "/usr/local/src/misp-modules/tools/update_misp_modules.sh" on my other installs, any guidance on this? I assume the install was done incorrectly.

EDIT2:
I figured out the problem with the git error, the misp-module install guide is flawed, and also the misp INSTALL.sh as well.
permissions are not set properly to the misp-module directory, hence why git does not work right.
I commented on this ticket re: the exact problem

#4045

So this is all solved now. Main problems were

1. Misp-module install procedure did not work.
2. needed to remove "-s" from the misp-module unit ExecStart for new modules to be recognized and not installed over.

Believe this can be closed now.

[righel]

Hello,
Probably was running with the wrong user:
sudo -u www-data /usr/local/src/misp-modules/tools/update_misp_modules.sh

Hope this helps.