New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multithread message authorization and decryption in Remoted #649

Merged
merged 10 commits into from May 23, 2018

Conversation

Projects
2 participants
@vikman90
Member

vikman90 commented May 22, 2018

This PR is related to issue #501.

Main features

  • Reload the file client.keys in a separate thread, and control the reloading frequency to avoid message decryption process starvation due to critical section blocking.
  • Implement a pool of threads to process message identification, decryption and decompression. This will take advantage of multi-core systems to speed up this process.

New options

<global>
  <queue_size>16384</queue_size>
</global>

queue_size

Size of the message input buffer in Analysisd (number of events).

Default value: 16384.
Minimum value: 1.
Recommended range: [16384..262144]

Internal options

remoted.worker_pool

This option defines the number of worker threads working in parallel between the payload reception (from the network) and the plain-text message delivering to Analysisd.

  • Default value: 4 threads.
  • Allowed values: an integer from 1 to 16.

remoted.keyupdate_interval

This option defines the minimum delay between keys file reloading.

  • Default value: 10 seconds.
  • Allowed values: an integer between 1 and 3600.

Dependencies

This PR requires PR #620 to work properly due to dependencies on external libraries.

@vikman90 vikman90 added this to To do in Wazuh 3.5 via automation May 23, 2018

@vikman90 vikman90 moved this from To do to In progress in Wazuh 3.5 May 23, 2018

@vikman90

This comment has been minimized.

Show comment
Hide comment
@vikman90

vikman90 May 23, 2018

Member

The domain socket between Remoted and Analysisd is blocking: when Analysisd runs slower than Remoted, the latter gets locked. There is no chance to discard the event once it calls send().

Solutions:

  • Set socket in non-blocking mode: simpler but it would also change this behavior in Logcollector, Syscheck, Modules, Monitor, Exec and Agent daemons.
  • Create input buffer with event dropping in Analysisd (#637)
Member

vikman90 commented May 23, 2018

The domain socket between Remoted and Analysisd is blocking: when Analysisd runs slower than Remoted, the latter gets locked. There is no chance to discard the event once it calls send().

Solutions:

  • Set socket in non-blocking mode: simpler but it would also change this behavior in Logcollector, Syscheck, Modules, Monitor, Exec and Agent daemons.
  • Create input buffer with event dropping in Analysisd (#637)

@vikman90 vikman90 referenced this pull request May 23, 2018

Closed

Segment and multithread Analysis daemon engine #637

4 of 4 tasks complete

@vikman90 vikman90 merged commit a72de47 into master May 23, 2018

Wazuh 3.5 automation moved this from In progress to Done May 23, 2018

@vikman90 vikman90 deleted the dev-remoted-multithread branch May 23, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment