CSV Import Connector getting hung up on imports

[rach24smith]

Description

We have noticed that the ImportCSV connector gets hung up on several occasions while we are trying to upload a csv of IOCs. Since it is built into the platform, we are unable to restart it like a normal connector, and have to restart the entire platform. When we restart the platform, the problem resolves.

Environment

1. OpenCTI version: 5.12.32 and past versions
2. OpenCTI client: frontend

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Upload a csv to the platform
2. run the importCSV connector on the document with a csv mapper that was already created

Expected Output

The expected result would be the import completes successfully

Actual Output

The import spins indefinitely, with no errors

Additional information

Screenshots (optional)

[Jipegien]

@rach24smith Does it concern a specific file, or files with a certain size? Are you able to share a problematic file with us for reproducing?

[rach24smith]

@Jipegien, it appears to be random. We have seen no pattern. The most recent time was on a CSV file with 600 lines in it, and it got hung up, but when we restarted the platform and tried again, it worked no issues. But we have seen it with much smaller files too!

[Jipegien]

Sorry I did not find a way to reproduce it. @Kedae @sbocahu any idea what could cause this or what could be retrieve to get more information?

[Jipegien]

It probably comes from worker queue congestion. Try to add some workers to be able to process the queue more rapidly.

I close the issue for now. If you have the problem again, please share config of your platform in term of worker, database and ram, as well as specifics about the file.

[rach24smith]

Hello @Jipegien, this happened again today, on one of our instances of OpenCTI that is barely used. The file only had 8 items in it for upload. It has 15 workers, CPU usage sits below 25%. Can you please reopen this ticket?

[Jipegien]

Thanks @rach24smith :) we are investigating.