Worker error when importing Network-Traffic object with nested properties

[AlexSanchezN]

We receive the error below when importing a Network-Traffic object via Worker from a stix bundle:

{'name': 'DATABASE_ERROR', 'message': 'Error in store update event'}

In the Worker logs we find the following error message:

ERROR:worker:{'name': 'DATABASE_ERROR', 'message': 'Error in store update event'} Traceback (most recent call last): File "/opt/opencti/./worker/worker.py", line 263, in data_handler self.api.stix2.import_bundle_from_json( File "/usr/local/lib/python3.10/dist-packages/pycti/utils/opencti_stix2.py", line 215, in import_bundle_from_json return self.import_bundle( File "/usr/local/lib/python3.10/dist-packages/pycti/utils/opencti_stix2.py", line 2421, in import_bundle self.import_observable(item, update, types) File "/usr/local/lib/python3.10/dist-packages/pycti/utils/opencti_stix2.py", line 1066, in import_observable self.opencti.stix_nested_ref_relationship.create( File "/usr/local/lib/python3.10/dist-packages/pycti/entities/opencti_stix_nested_ref_relationship.py", line 266, in create result = self.opencti.query( File "/usr/local/lib/python3.10/dist-packages/pycti/api/opencti_api_client.py", line 348, in query raise ValueError( ValueError: {'name': 'DATABASE_ERROR', 'message': 'Error in store update event'}

No associated error is found in the Platform logs

After some testing with the code that generates the bundles, we find:

The problem is related to the network-traffic object.
It only appears if we add src_ref, start or end properties. (we do not add dst_ref, haven’t checked the case).
If we create the object without those properties, it is created correctly.
If we only add start and end properties, the object is created correctly.
If we only add src_ref, most objects are created correctly, but the error still appears (let’s say 20% of the time).
If we add src_ref, start and end, ALL network-traffic object creation fails.

Environment

Ubuntu 22.04 baremetal installation. All platform components in different VMs.
Current platform version 5.12.32

Reproducible Steps

Difficult to reproduce as the same bundle that gives the error when importing via Worker, does not give an error when imported via WorkBench

Expected Output

A nice network-traffic object with associated src_ref, start and end time.

Actual Output

The errors above

Additional information

There are similar reports in Filigran's Slack, posted after ours:
https://filigran-community.slack.com/archives/C06CF1N302W/p1708432322249959

[labo-flg]

Hi @AlexSanchezN and thanks for the details!

After some investigations, I narrowed down the issue to a simple repro case:

• Create an observable > network traffic, with a start and end.
• Open it, go to knowledge, and add a nested object
• select an ipv4 address, and create a "src" relationship
• --> store update event
• reload the details page of the observable > the relationship is still created successfully

The error prevents the modification to be published in the stream (as you can see if you do this operation with a <platformurl>/stream tab opened to look at the raw event stream -> you see the creation of the Network Traffic but nothing about the relationship).

However, the data is still ingested correctly. The only side effect I see would be that if the event is not published, the platform cannot trigger automatic enrichments or any scripted process relying on the event stream.

Could you confirm this on your side ? Do you get the right data object ingested, with the src relationship ?

I'll try to fix this asap.

[AlexSanchezN]

Hi @labo-flg , thanks for your help.
I've just tried from the UI, and it is exactly as you say.

Our real problem is from the Worker as once we receive the store update event error, nothing else in the bundle is ingested/created.
I suppose that when the error is resolved, everything will be ingested normally.

Great work! Thanks again!