Skip to content

Commit

Permalink
push receiver doc update to dev. already merged to 2.4 branch
Browse files Browse the repository at this point in the history
  • Loading branch information
m0duspwnens committed Apr 12, 2024
1 parent 51f1a8e commit f8597b3
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,8 @@ Each receiver node runs :ref:`logstash` and :ref:`redis` and allows for events t

If you don't have any receiver nodes and the manager goes down, the search nodes do not index anything because they cannot connect to :ref:`redis`. The agents cannot connect to :ref:`logstash` so the pipeline starts backing up on the agents. In this same scenario with a receiver node the agents would not be able to talk to :ref:`logstash` on the manager and then would try to connect to the receiver node. Once connected they would send their logs to the receiver like nothing was wrong. The search nodes connect to both the manager and receiver nodes and pull events from the :ref:`redis` queue. If the manager goes down, the search nodes will keep pulling the log events from the queue on the receiver node. This allows for scaling of the pipeline. More receivers + more search nodes = more event ingestion volume.

If you have a manager or managersearch that is under heavy load due to handling a high volume of events, then system resources can be freed by directing the Elastic Agent to only output events to the receiver node(s) in the environment. Once all configurable and advanced settings are enabled, this feature can be set in SOC Configuration UI under ``elasticfleet > enable_manager_output``. Setting this to ``False`` will prevent the Elastic Agent from sending events to the manager, managersearch, or standalone nodes.

Receiver nodes need to be close to the search nodes because when you add a new receiver node to the grid, the search nodes add the :ref:`redis` service as an input in their configs automatically. If you were to place a receiver node at a remote site, then ALL of your search nodes would be trying to access that :ref:`redis` queue remotely. You do not save any bandwidth by placing a receiver node at a remote site.

There are a couple of things to be aware of regarding receiver nodes and Elastic Agents. The first is Fleet which handles things like updating the agents and scheduling searches. The other is the Elastic Agent log output, which in this case is :ref:`logstash` running on the manager or receiver node. Due to limitations in Elastic licensing we can only have a single output policy. That means that when you add a receiver or a fleet node it gets added to a list that is distributed to the agents. The agents go down that list and stop after a successful connection. The only way to direct agents to specific receivers is to use firewall rules to block agents to certain receivers. Again keep in mind that there is no bandwidth savings here because the search nodes still need to empty the :ref:`redis` queue on the receiver nodes.
Expand Down

0 comments on commit f8597b3

Please sign in to comment.