SPIKE - Enhancing agent communication setup

[TomasTurina]

Description:

As detailed in wazuh/wazuh#22677, Wazuh's current communication setup is complex and needs to be refactored.

The focus of this issue will be on the following tasks:

• Initial client design, including technological and library research.
• PoC implementation for agent.

The following items must be taken into account to execute this task:

• Conduct initial research on system design and relevant technologies.
• Collaborate with the server team to align on communication protocols and API integration.
• Explore client-side designs and technologies for seamless interaction with the Agent Comms API.
• Work on a Proof of Concept implementation to validate communication flows between agents and servers.

[TomasTurina]

Some considerations

Registration of a new agent

• Whoever installs an agent has to register the agent on the server only once.
• The UUID is chosen by the agent through the new CLI.
• Use the API (port 55000), passing username and password to it.
• Server management API.

Agent authentication and authorization

• To send events the agent needs a token.
• For the request it will use its UUID.
• When it expires it must request another one and so on.
• oAuth 2.0 with JWTs.
• Agent comms API.

Communication agent -> server

• Asynchronous and event driven.
• Open connection, pass token, send events, close connection.
• Implement batching.
• HTTPS, API REST.
• Agent comms API.

Communication server -> agent

• Connection-oriented.
• The connection is the agent's initiative but kept by the manager.
• The agent would pull to see if it has anything.
• HTTPS 2, WebSockets.
• Agent comms API.

API endpoints

• /login to authenticate (request token).
• /stateless to send events, we pass the UUID, the token (Bearer...) and the events.
• /stateful the same as the previous one, but it requires persistent data.
• /commands in opposite direction, request made by agent to manager.

[TomasTurina]

Tasks

Investigation

• Technologies within our reach.
  • C++ 17, CMake, everything as standard as possible.
• Look for a library that has everything.
  • Avoid one for HTTPS, one for WebSockets, ...
• Describe the behavior of the new agent.
  • Create UML designs.
• Define new building system.
• Define new repository structure.

POC

• Coordinate with the server team.
  • Initially work with mocks.
• Establish connection, run endpoints.
  • A few lines of code.
• Upload it to a /poc folder in the new repository.
  • A program that works, must connect to the server POC.

[jr0me]

Initial libraries investigation

There are several C++ libraries that support our requirements. The ones described in this small list are widely used in the industry and tested. Note that no single library covers all needs: those that support high-level protocols like HTTP and HTTP/2 often lack native support for UDP, while libraries handling low-level protocols may not provide comprehensive support for higher-level abstractions.

According to wazuh/wazuh#23395 the server team is also considering a number of libraries, we should encourage sharing the same technology across both Agent and Server.

Libraries for Network Connections:

ZeroMQ (ØMQ):
ZeroMQ (ØMQ) is a high-performance asynchronous messaging library designed for distributed and concurrent applications. It supports various messaging patterns like publish/subscribe, request/reply, and client/server over multiple transports, including TCP, UDP, in-process, inter-process, multicast, and WebSocket (does not natively support HTTP/HTTP2). Highly suitable for building scalable and resilient distributed systems. The libzmq library is licensed under the Mozilla Public License 2.0.

https://zeromq.org/get-started/

gRPC:
gRPC is a high-performance open-source RPC framework that facilitates seamless communication between client and server applications, making it ideal for distributed systems. It uses HTTP/2 and Protocol Buffers for efficient serialization (does not support raw TCP/UDP). gRPC supports multiple languages, allowing interoperability between different systems. It is designed for low-latency and scalable communication, with features like load balancing, tracing, and authentication. It can also be extended to support various data formats like JSON, Protobuf, and others. It's license is Apache 2.

https://grpc.io/

Boost.Asio / Boost.Beast:
Boost.Asio provides a comprehensive suite for asynchronous network programming, supporting TCP, UDP, and serial communication. Boost.Beast extends Boost.Asio with HTTP and WebSocket capabilities. These libraries are highly performant and integrate well with other Boost libraries, offering extensive functionality for complex networking tasks.

https://www.boost.org/users/license.html
https://www.boost.org/doc/libs/1_84_0/doc/html/boost_asio.html
https://www.boost.org/doc/libs/1_85_0/libs/beast/doc/html/index.html

ASIO (Standalone):
The standalone ASIO library offers asynchronous I/O capabilities without the Boost dependency. It supports TCP, UDP, and serial port communications, and is suitable for applications requiring high performance (requires additional libraries for HTTP/HTTP2). ASIO is lightweight and provides efficient network communication primitives.

https://www.boost.org/LICENSE_1_0.txt
https://think-async.com/Asio/

Libraries for JWTs:

jwt-cpp:
jwt-cpp is a lightweight library for creating and verifying JSON Web Tokens (JWT). It supports various algorithms for signing and verifying tokens, making it suitable for implementing token-based authentication in C++ applications. jwt-cpp is easy to integrate and provides a straightforward API for handling JWTs. Licensed under MIT.

https://thalhammer.github.io/jwt-cpp/