STIX or CyBox library not installed correctly #1669

santi10 · 2016-11-15T16:11:59Z

Hi,

I have an issue with Cybox and STIX, this is the message what I see, such as:

_Mitre's STIX and Cybox python libraries have to be installed in order for MISP's STIX export to work. Make sure that you install them (as described in the MISP installation instructions) if you receive an error below.
If you run into any issues here, make sure that both STIX and CyBox are installed as described in the INSTALL.txt file. The required versions are:
STIX:
CyBox:
Other versions might work but are not tested / recommended.

STIX and Cybox libraries....STIX or CyBox library not installed correctly_

This is my version Cybox and Stix
STIX: 1.1.1.4
CyBox: 2.1.0.12

So I was installing it and I wasn't anything issue.

Can you help me, please? Let me know if you need more info.

Regards!

santi10 · 2016-11-17T11:15:11Z

Hi,

Have you any idea about this issue, please?

Thanks!

adulau · 2016-11-17T11:23:24Z

Hi,

Have you done the standard installation?

Here is the procedure:

# install Mitre's STIX and its dependencies by running the following commands:
sudo apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev python-setuptools
cd /var/www/MISP/app/files/scripts
sudo -u www-data git clone https://github.com/CybOXProject/python-cybox.git
sudo -u www-data git clone https://github.com/STIXProject/python-stix.git
cd /var/www/MISP/app/files/scripts/python-cybox
sudo -u www-data git checkout v2.1.0.12
sudo python setup.py install
cd /var/www/MISP/app/files/scripts/python-stix
sudo -u www-data git checkout v1.1.1.4
sudo python setup.py install

Let us know if you have any issue.

santi10 · 2016-11-17T14:57:53Z

Hi Adulau,

Yes, I did it so. I don't undestand why STIX and CyBOX are not working on MISP.

I don't know that do...I'm desperate with this!

Thx

adulau · 2016-11-17T15:03:12Z

If you go to the "Download as..." of an event. Have you tried the STIX JSON or XML export?

iglocska · 2016-11-17T15:09:28Z

Could you run the following line and post the results here?

python /var/www/MISP/app/files/scripts/stixtest.py

Thanks!

santi10 · 2016-11-18T09:10:11Z

Hi,

The result is "{"cybox": "2.1.0.12", "stix": "1.1.1.4", "success": 1}

When I go to "Download as" and select the format "STIX XML" this is the result:

This XML file does not appear to have any style information associated with it. The document tree is shown below.

An Internal Error Has Occurred.
An Internal Error Has Occurred.
/events/stix/download/16/true.xml

STIX and Cybox doesn't want work!

Thx

iglocska · 2016-11-18T12:48:12Z

Could you check the file permissions for the python scripts in /var/www/MISP/app/files/scripts/ ? It sounds like MISP may not have access to them.

santi10 · 2016-11-21T09:00:14Z

Hi,

Yes, these are:

I think that is fine...

Thx

FloatingGhost · 2016-11-21T09:02:28Z

An Internal Error Has Occurred.
An Internal Error Has Occurred.

Any funny business going on in /var/www/MISP/app/tmp/logs/error.log?

santi10 · 2016-11-21T09:08:58Z

Yes, you can see it

santi10 · 2016-11-28T18:01:45Z

Do you any idea about this, please??

iglocska · 2016-11-28T18:16:59Z

Could you run the following to ensure that it's not a permission issue with your STIX installation?

sudo -H -u www-data bash -c 'python /var/www/MISP/app/files/scripts/stixtest.py'

santi10 · 2016-11-29T08:33:51Z

Hi Iglocska,

This is the result:

Seem that is fine, ok???

Thx

iglocska · 2016-11-29T09:36:40Z

Hmm it is. Weird. So you get that with the apache user, but misp fails to ruin the same diagnostic. That's odd. Will look into what else could cause this.

…

On Nov 29, 2016 9:33 AM, "santi10" ***@***.***> wrote: Hi Iglocska, This is the result: [image: image] <https://cloud.githubusercontent.com/assets/6546107/20702141/e372c872-b616-11e6-8ea5-a6fec4961fc8.png> Seem that is fine, ok??? Thx — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1669 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADf6wJhPx6D0QzNne3t3KJoaPlGa9-Zeks5rC-NxgaJpZM4KytkP> .

santi10 · 2016-11-29T11:06:42Z

Yes, its very odd. I don't know to must do really. I hope that you can help me with this.

Regards

iglocska · 2016-11-29T11:57:38Z

We are currently abroad giving a workshop, but will look at this as soon as I get back.

…

On Tue, Nov 29, 2016 at 12:06 PM, santi10 ***@***.***> wrote: Yes, its very odd. I don't know to must do really. I hope that you can help me with this. Regards — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1669 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADf6wMaSvAgtMkrXcXXJZJog_KAvaa6Yks5rDAdDgaJpZM4KytkP> .

santi10 · 2016-11-29T15:42:53Z

Great, thanks a lot

FloatingGhost · 2016-11-30T09:40:31Z

hokay, I'm here and python's my thing ;)

Can you run a thing for me?

sudo -u www-data curl https://gist.githubusercontent.com/anonymous/459d0ac5d75dba2092cf6e90fd72770d/raw/165d2aa121c361fc007641c1654eefb38fb2ad3f/misp2stix.py   -o /var/www/MISP/app/files/scripts/misp2stix.py

Then try downloading the STIX file. It'll still error, but there'll be STIX_DEBUG.log in /var/www/MISP/app/webroot

Paste the contents of that file here.

santi10 · 2016-12-01T10:17:33Z

Hi FloatingGhost!

I've done that but I've the same reults.

You script has the line "logfile = "STIX_DEBUG.log", I've modified this line with "logfile = "/tmp/STIX_DEBUG.log".

I created this file "STIX_DEBUG.log" on /tmp/ with the owner www-data and permissions 777.

When I donwload the event in format STX I see the same error.

This XML file does not appear to have any style information associated with it. The document tree is shown below.

An Internal Error Has Occurred.
An Internal Error Has Occurred.
/events/stix/download/18.xml

The file "STIX_DEBUG.log" isn't working because I don't see nothing on this file, no events.

Sorry but I'm desperate

Thanks for you help!

FloatingGhost · 2016-12-05T01:24:47Z

You see nothing? Hm, that seems to imply the libraries aren't installed.

Tell me, if you open a python prompt and enter

import sys, json, uuid, os, time, datetime, re
from misp2cybox import *
from misp2ciq import *
from dateutil.tz import tzutc
from stix.indicator import Indicator
from stix.indicator.valid_time import ValidTime
from stix.ttp import TTP, Behavior
from stix.ttp.malware_instance import MalwareInstance
from stix.incident import Incident, Time, ImpactAssessment, ExternalID, AffectedAsset
from stix.exploit_target import ExploitTarget, Vulnerability
from stix.incident.history import JournalEntry, History, HistoryItem
from stix.threat_actor import ThreatActor
from stix.core import STIXPackage, STIXHeader
from stix.common import InformationSource, Identity, Confidence
from stix.data_marking import Marking, MarkingSpecification
from stix.extensions.marking.tlp import TLPMarkingStructure
from stix.common.related import *
from stix.common.confidence import Confidence
from stix.common.vocabs import IncidentStatus
from cybox.utils import Namespace

is it happy?

santi10 · 2016-12-13T09:48:30Z

Hi,

Give me some error!!

This XML file does not appear to have any style information associated with it. The document tree is shown below.

An Internal Error Has Occurred.
An Internal Error Has Occurred.
/events/stix/download/19.xml

I don't konw to do, I'm odd!

Regards!

santi10 · 2016-12-13T10:02:40Z

I haven't had any error with python...Do you have some idea about this??

Thx

iglocska · 2016-12-13T10:33:05Z

If you get an internal error, could you check your MISP error log for the stack trace?

It's located in:

/var/www/MISP/app/tmp/logs/error.log

santi10 · 2016-12-13T10:38:51Z

Hi,

This is the message

What I can do, please?

Thx

iglocska · 2016-12-13T10:40:26Z

Hmph so it's indeed the python part that's failing, back to square one. You've tried what @FloatingGhost suggested and see if the imports fail in a python script, right?

santi10 · 2016-12-13T10:42:50Z

Yes, I did that and I haven't had any error.

santi10 · 2016-12-13T10:47:56Z

In Server Settings -> Diagnostic, I may see this:

STIX and Cybox libraries....STIX or CyBox library not installed correctly

iglocska · 2016-12-13T11:00:00Z

OK, let's try something dirty, running @FloatingGhost from MISP, let's see if it's a permission issue for the apache user.

I've pushed a new branch that has a really dirty test script in it. Simply switch to that branch:

git checkout origin/stixtest

After that, log in as a site administrator and via the url bar go to:

https://my_misp_instance/servers/test

Could you paste the output from the top of the screen? It's really dirty and quick but should give us the feedback we need.

iglocska · 2016-12-13T11:00:18Z

Oh and once you're done simply switch back to the 2.4 branch by typing:

git checkout 2.4

santi10 · 2016-12-13T15:35:45Z

I modified the permisson of cybox and stix again, its works fine.

STIX and Cybox libraries

Mitre's STIX and Cybox python libraries have to be installed in order for MISP's STIX export to work. Make sure that you install them (as described in the MISP installation instructions) if you receive an error below.
If you run into any issues here, make sure that both STIX and CyBox are installed as described in the INSTALL.txt file. The required versions are:
STIX: 1.1.1.4
CyBox: 2.1.0.12
Other versions might work but are not tested / recommended.

STIX and Cybox libraries....OK
STIX library version....OK
CYBOX library version....OK

But I follow with the same error when I go to dolwnload STIX format

santi10 · 2016-12-13T15:42:01Z

I've seen this

is it can be the problem???

iglocska · 2016-12-13T15:44:30Z

Are you downloading an individual event in stix format? So event view -> download as -> STIX?

Or are you trying to generate a full export via the export view? It looks like you don't have background workers running, perhaps?

santi10 · 2016-12-13T15:48:11Z

I'm trying only individual but I did it for full experto a long time too.

iglocska · 2016-12-13T15:51:59Z

Individual should be fine and the cache job shouldn't affect it then.

Could you create a new event with a single ip-dst attribute, publish it and see if you can export that as STIX?

santi10 · 2016-12-13T15:53:51Z

Yes, I did it too and I had the same error.

This XML file does not appear to have any style information associated with it. The document tree is shown below.

An Internal Error Has Occurred.
An Internal Error Has Occurred.
/events/stix/download/25/true.xml

santi10 · 2016-12-13T15:54:13Z

An Internal Error Has Occurred. An Internal Error Has Occurred. /events/stix/download/25/true.xml

santi10 · 2016-12-13T15:54:40Z

santi10 · 2016-12-13T15:56:40Z

How I can kill all the process that these are in Queued????

iglocska · 2016-12-13T15:59:39Z

administration -> server settings -> workers

Each queue should have a counter on pending jobs and a trash icon. Click the trash icon to clear a queue.

santi10 · 2016-12-13T16:05:09Z

Yes, I know that but If I delete the icon I can see as the jobs are on queued go to run again...I want to kill for fully, is it possible?

iglocska · 2016-12-13T16:06:09Z

They are killed, don't worry about them. Think of the jobs screen as a somewhat interactive log of jobs.

santi10 · 2016-12-13T16:10:06Z

Ok, great.

I follow with the issue, do you think that it has fix? I want to dowonload STIX format please!!

Rafiot · 2016-12-13T16:12:12Z

STIX export works on other instances, so yes, it has a fix.

FloatingGhost · 2016-12-13T16:14:14Z

Honestly I think you've managed to... somehow have a messed up OS. This shouldn't happen at all

santi10 · 2016-12-13T16:15:06Z

I want to believe you but with the help the folks, we can't fix it.
I would like to know it but I see all ok (I think so), I'm desperate

iglocska · 2016-12-13T16:15:53Z

Hold on, working on some additional test scripts so we can debug what's going wrong there.

santi10 · 2016-12-13T16:17:19Z

My OS is Ubuntu 10.04.5 LTS, I follow this manual for the installation of MISP

https://github.com/MISP/MISP/blob/2.4/INSTALL/INSTALL.ubuntu1404.txt

FloatingGhost · 2016-12-13T16:17:20Z

I want to believe you but with the help the folks, we can't fix it.

It works on mine, it's not something broken with the code itself

FloatingGhost · 2016-12-13T16:17:26Z

wait. Ubuntu 10!?

santi10 · 2016-12-13T16:19:37Z

Do you recommend other version???

iglocska · 2016-12-13T16:20:24Z

16.04 for sure! 10.04 is ancient. You sure it isn't a typo?

FloatingGhost · 2016-12-13T16:20:26Z

At least 14, ideally 16. 10 is, well, 6 years old. It's barely even supported

Rafiot · 2016-12-13T16:21:50Z

The install procedure you followed is for 14.04. And on new install, you should be using 16.04. And this install procedure: https://github.com/MISP/MISP/blob/2.4/INSTALL/INSTALL.ubuntu1604.txt

santi10 · 2016-12-13T16:22:43Z

ok, I make it right now

Rafiot · 2016-12-13T16:30:45Z

Closing then.

santi10 · 2016-12-14T11:09:46Z

Hi!!

I've just downloaded the STIX format!!

the issue is the operation system version, Ubuntu 16.04 is working properly!

Thanks a lot to everybody!

iglocska · 2016-12-14T14:17:12Z

Excellent news, thanks for the feedback!

adulau added the T: support Type: support. This issue is seeking support on a problem or question label Nov 17, 2016

Rafiot closed this as completed Dec 13, 2016

jdubp mentioned this issue May 11, 2017

STIX download returning An Internal Error Has Occurred. #2181

Closed

STIX or CyBox library not installed correctly #1669

STIX or CyBox library not installed correctly #1669

Comments

santi10 commented Nov 15, 2016

santi10 commented Nov 17, 2016

adulau commented Nov 17, 2016

santi10 commented Nov 17, 2016

adulau commented Nov 17, 2016

iglocska commented Nov 17, 2016

santi10 commented Nov 18, 2016

iglocska commented Nov 18, 2016

santi10 commented Nov 21, 2016

FloatingGhost commented Nov 21, 2016

santi10 commented Nov 21, 2016

santi10 commented Nov 28, 2016

iglocska commented Nov 28, 2016

santi10 commented Nov 29, 2016

iglocska commented Nov 29, 2016 via email

santi10 commented Nov 29, 2016

iglocska commented Nov 29, 2016 via email

santi10 commented Nov 29, 2016

FloatingGhost commented Nov 30, 2016

santi10 commented Dec 1, 2016

FloatingGhost commented Dec 5, 2016

santi10 commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

santi10 commented Dec 13, 2016

santi10 commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

Rafiot commented Dec 13, 2016

FloatingGhost commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

santi10 commented Dec 13, 2016

FloatingGhost commented Dec 13, 2016

FloatingGhost commented Dec 13, 2016

santi10 commented Dec 13, 2016

iglocska commented Dec 13, 2016

FloatingGhost commented Dec 13, 2016

Rafiot commented Dec 13, 2016 • edited

santi10 commented Dec 13, 2016

Rafiot commented Dec 13, 2016

santi10 commented Dec 14, 2016

iglocska commented Dec 14, 2016

Rafiot commented Dec 13, 2016 •

edited