Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create agent database files from template atomically #23216

Merged
merged 3 commits into from
May 2, 2024

Conversation

vikman90
Copy link
Member

@vikman90 vikman90 commented May 2, 2024

Related issue
Closes #22847

This pull request aims to address a fault tolerance issue in wazuh-db, which may result in empty database files if abruptly closed. This scenario can occur in an environment where the service takes longer than a minute to shut down, as it forcibly terminates wazuh-db with SIGKILL.

Proposed solution

The proposed solution entails:

  • Creating temporary files and renaming them to the database name once they are filled.
    • If left empty, they will be regenerated upon the next execution.
  • Adding a log entry confirming the proper closure of wazuh-db.

Not considered

  • Validation of databases once created, or upon each opening.
  • Cleanup of potential stray temporary files.

Tests

  • Unit tests for the modified function: wdb_create_agent_db2.
  • Manual test: abruptly closing wazuh-db while it is creating databases.

Manual test

for i in {1..1000}; do echo "agent $i begin"; done | ~/wazuh-tools/utils/wdb-query.py & (sleep 0.1; pkill -9 wazuh-db)
Before
-rw-r----- 1 wazuh wazuh      0 May  2 09:51 187.db
-rw-r----- 1 wazuh wazuh      0 May  2 09:51 188.db
-rw-r----- 1 wazuh wazuh      0 May  2 09:51 189.db
-rw-r----- 1 wazuh wazuh      0 May  2 09:51 190.db
-rw-r----- 1 wazuh wazuh      0 May  2 09:51 191.db
-rw-r----- 1 wazuh wazuh      0 May  2 09:51 192.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 193.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 194.db
-rw-r----- 1 wazuh wazuh      0 May  2 09:51 195.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 196.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 197.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 198.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 199.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 200.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 201.db
-rw-r----- 1 wazuh wazuh  20480 May  2 09:51 202.db
After
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 076.db
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 077.db
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 078.db
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 080.db
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 081.db
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 082.db
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 083.db
-rw-r----- 1 wazuh wazuh 240K May  2 13:20 086.db
-rw-r--r-- 1 wazuh wazuh    0 May  2 13:20 087.db.new
-rw-r--r-- 1 wazuh wazuh    0 May  2 13:20 090.db.new
-rw-r----- 1 wazuh wazuh 212K May  2 13:20 global.db
srw-rw---- 1 wazuh wazuh    0 May  2 13:20 wdb

@vikman90 vikman90 self-assigned this May 2, 2024
@vikman90 vikman90 requested a review from cborla May 2, 2024 11:35
@vikman90 vikman90 added type/bug Something isn't working module/db Wazuh DB engine labels May 2, 2024
@vikman90
Copy link
Member Author

vikman90 commented May 2, 2024

cborla
cborla previously approved these changes May 2, 2024
Copy link
Member

@cborla cborla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@LucioDonda LucioDonda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It LGTM but in my opionion we could add those little details

src/wazuh_db/wdb.c Show resolved Hide resolved
LucioDonda
LucioDonda previously approved these changes May 2, 2024
@vikman90 vikman90 dismissed stale reviews from LucioDonda and cborla via 14e9785 May 2, 2024 14:25
@vikman90 vikman90 force-pushed the fix/22847-broken-database branch from 2567a2a to 14e9785 Compare May 2, 2024 14:25
Copy link
Member

@cborla cborla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vikman90 vikman90 merged commit dd832e5 into 4.8.0 May 2, 2024
57 of 60 checks passed
@vikman90 vikman90 deleted the fix/22847-broken-database branch May 2, 2024 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module/db Wazuh DB engine type/bug Something isn't working
Projects
None yet
3 participants