No MISP Standard ZMQ Green Icon on MISP Dashboard

[faustus25]

Still having issues running ZMQ on MISP dashboard to publish events from MISP to the MISP-Dashboard.
Each update has either ran ZMQ temporarily or not at all i.e. the Standard ZMQ Icon is not present on the default MISP-dashboard landing page.

git describe --tags
v1.1-2-gce26610

ZMQ is active and running on MISP

ZeroMQ Server Status

Start time: 2018/10/12 11:33:30
Settings read at: 2018/10/12 10:49:24
Events processed: 13

ZMQ Listening:

0.0.0.0:50000 2473/python3

Redis-server listening on port 6250 TCP

0.0.0.0:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
127.0.0.1:6250 1405/redis-server
:::6250 1405/redis-server

How can I check to ensure the ZMQ is running correctly for the MISP-Dashboard?

[faustus25]

Config:

[Dashboard]
#hours
graph_log_refresh_rate = 1
#sec
rotation_wait_time = 30
max_img_rotation = 10
hours_spanned = 48
zoomlevel = 15
maxCacheHistory = 30
# [1->12]
size_dashboard_left_width = 5
size_openStreet_pannel_perc = 55
size_world_pannel_perc = 35
item_to_plot = Attribute.category
fieldname_order=["Event.id", "Attribute.Tag", "Attribute.category", "Attribute.type", ["Attribute.value", "Attribute.comment"]]
char_separator=||

[GEO]
#min
updateFrequency = 60
zoomlevel = 11
# ~meter
clusteringDistance = 10

[CONTRIB]
max_number_of_last_contributor = 10
min_between_reload = 5
additional_help_text = ["Sightings multiplies earned points by 2", "Editing an attribute earns you the same as creating one"]

[Log]
directory=logs
filename=logs.log

[RedisGlobal]
host=localhost
port=6250
#misp_web_url = http://192.168.56.50
misp_web_url = 'https://misp/'
#zmq_url=tcp://192.168.56.50:50000
zmq_url=tcp://localhost:50000

[RedisLIST]
db=3
listName=bufferList

[RedisLog]
db=0
streamLogCacheKey = streamLogCache
streamMapCacheKey = streamMapsCache
channel=1
channelLastContributor = lastContributor
channelLastAwards = lastAwards

[RedisMap]
db=1
channelDisp=PicToDisplay
# Database updates regularly make sure to adapt date
pathMaxMindDB=/usr/local/src/misp-dashboard/data/GeoLite2-City_20180807/GeoLite2-City.mmdb
path_countrycode_to_coord_JSON=/usr/local/src/misp-dashboard/data/country_code_lat_long.json

[RedisDB]
db=2

[mokaddem]

Probably a missing link.
What you can do is to test the chain.

• MISP has a tool to check if ZMQ is working the intended way sub.py
• You can check if the dashboard receives the feed by printing the content variable on the subscriber script

Can you check?

[faustus25]

MISP sub.py traceback: (all good)

{"uptime": 15340, "status": "While you're dying I'll be still alive."}
{
    "Log": {
        "model_id": "5088178",
        "description": "Attribute \"api.bing.com|13.107.5.80\" (5088178) added by User \"SYSTEM\" (0).",
        "action": "add",
        "change": "to_ids () => (1), distribution () => (5), type () => (domain|ip), category () => (Network activity), uuid () => (xxxx-xxxx), event_id () => (4678), value1 () => (api.bing.com), value2 () => (13.107.5.80)",
        "changes": 8,
        "title": "Attribute (5088178) from Event (4678): Network activity\/domain|ip api.bing.com",
        "model": "Attribute",
        "user_id": 0,
        "email": "SYSTEM",
        "org": "SYSTEM",
        "created": "2018-10-12 15:05:04"
    },
    "action": "log"
}
{"uptime": 15350, "status": "And when you're dead I will be still alive."}
{"uptime": 15360, "status": "And believe me I am still alive."}
{"uptime": 15370, "status": "I'm doing science and I'm still alive."}
{"uptime": 15380, "status": "I feel FANTASTIC and I'm still alive."}
{"uptime": 15390, "status": "While you're dying I'll be still alive."}
{"uptime": 15400, "status": "And when you're dead I will be still alive."}

I am launching the ./start_all.sh at reboot as part of a cronjob.

Tried starting it by the cmd line and get this on the zmq_dispatcher.py:

Traceback (most recent call last):
  File "./zmq_dispatcher.py", line 299, in <module>
    main(args.sleeptime)
  File "./zmq_dispatcher.py", line 274, in main
    process_log(zmqName, content)
  File "./zmq_dispatcher.py", line 256, in process_log
    dico_action[topic](zmq_name, jsonevent)
  File "./zmq_dispatcher.py", line 230, in handler_attribute
    geo_helper.getCoordFromIpAndPublish(jsonattr['value'], jsonattr['category'])
  File "/usr/local/src/misp-dashboard/helpers/geo_helper.py", line 113, in getCoordFromIpAndPublish
    self.push_to_redis_geo(self.keyCategRad, coord['lon'], coord['lat'], json.dumps(ordDic))
  File "/usr/local/src/misp-dashboard/helpers/geo_helper.py", line 184, in push_to_redis_geo
    self.serv_redis_db.geoadd(keyname, lon, lat, content)
  File "/usr/local/src/misp-dashboard/DASHENV/lib/python3.5/site-packages/redis/client.py", line 2121, in geoadd
    return self.execute_command('GEOADD', name, *values)
  File "/usr/local/src/misp-dashboard/DASHENV/lib/python3.5/site-packages/redis/client.py", line 668, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/usr/local/src/misp-dashboard/DASHENV/lib/python3.5/site-packages/redis/client.py", line 680, in parse_response
    response = connection.read_response()
  File "/usr/local/src/misp-dashboard/DASHENV/lib/python3.5/site-packages/redis/connection.py", line 629, in read_response
    raise response
redis.exceptions.ResponseError: unknown command 'GEOADD'

Above error related this know issue:

https://github.com/MISP/misp-dashboard/issues/1

[mokaddem]

For the zmq_dispatcher.py, have you tried the fix (use a redis version supporting the GEOADD command >=3.2.0)? Did it solves your problem or do you have another error message?

[faustus25]

Problem resolved by commenting out line 184 in geo_helper.py

def push_to_redis_geo(self, keyCateg, lon, lat, content):
now = datetime.datetime.now()
today_str = util.getDateStrFormat(now)
keyname = "{}:{}".format(keyCateg, today_str)
#self.serv_redis_db.geoadd(keyname, lon, lat, content)
self.logger.debug('Added to redis: keyname={}, lon={}, lat={}, content={}'.format(keyname, lon, lat, content))

[yixiangbo]

I have issue on the same you. Can you help me fix it