Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Events are uploaded twice using timesketch_import_client with csv or jsonl files. #2334

Closed
S-Nicholas opened this issue Sep 19, 2022 · 4 comments
Assignees
Labels
Bug Data import All things that are with importing data
Milestone

Comments

@S-Nicholas
Copy link

S-Nicholas commented Sep 19, 2022

Describe the bug
Events are uploaded twice using timesketch_import_client with csv or jsonl files.

To Reproduce
Steps to reproduce the behavior:

  1. Generate csv or json file
  2. Use timesketch_importer.py to import the file
  3. Events are imported twice in timesketch

Expected behavior
Events should be imported only once.

Additional context
The issue is located in the ImportStreamer class.
Private property _data_lines is not empty after calling the flush method.
When the ImportStreamer object is destroyed, flush method is called again.
Fix: ImportStreamer._upload_data_buffer method should call _reset method in case of successfull data upload.

@jaegeral jaegeral added this to the Q3 milestone Mar 28, 2023
@jaegeral jaegeral added the Data import All things that are with importing data label Mar 28, 2023
@jaegeral
Copy link
Collaborator

This def needs to be reproduced.

Hope we can fix it in Q3.

@S-Nicholas
Copy link
Author

Same issue as #2796

@berggren
Copy link
Contributor

I cannot seem to reproduce this. I'm importing a CSV with 48300 events with the importer client and I get the expected result.

@S-Nicholas Do you have a stable reproduce?

@berggren berggren self-assigned this Sep 19, 2023
@jaegeral
Copy link
Collaborator

jaegeral commented Nov 8, 2023

I am closing this in favour to #2796

@jaegeral jaegeral closed this as completed Nov 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Data import All things that are with importing data
Projects
None yet
Development

No branches or pull requests

3 participants