default group merged.mg error

[IOTech17]

Wazuh version	Component	Install type	Install method	Platform
4.7.0	Group management	Manager	officiel docker image	n/a

I have upgrade from Wazuh docker 4.5.2 to 4.60 then from 4.6.0 to 4.7.0

When checking the merged.mg file of the default group I get an error that the file contains an error

error on line 1 at column 847: xmlParseEntityRef: no name

Is this a bug or normal?

Another question is there a way to make sure any news group created get all the files the default has?

Thank you in advance,

[rauldpm]

Investigated in wazuh/wazuh-docker#1154

Reproduced through a 4.7.1 Wazuh installation assistant deployment, not related to Wazuh docker, added to @wazuh/core initially to triage it

Thanks @IOTech17

[Selutario]

Update

After taking a look at the code and with the help of @Desvelao, these are the conclusions:

The dashboard is sending a GET /groups/default/files/merged.mg/xml API request to fetch the XML content of the merged.mg file. However, only agent.conf files are returned as XML. All others are parsed to JSON:

wazuh/framework/wazuh/core/configuration.py

Lines 819 to 829 in 037ede2

	else:
	if filename == "agent.conf":
	data = get_agent_conf(group_id, limit=None, filename=filename, return_format=return_format)
	elif filename == "rootkit_files.txt":
	data = _rootkit_files2json(file_path)
	elif filename == "rootkit_trojans.txt":
	data = _rootkit_trojans2json(file_path)
	elif filename == "ar.conf":
	data = _ar_conf2json(file_path)
	else:
	data = _rcl2json(file_path)

There is an XML validator in the dashboard, which is the one that shows the error message since the content obtained is JSON format.

Proposal

As I mentioned, except for the agent.conf files, all others are transformed into JSON even if the GET /groups/{group_id}/files/{file_name}/xml endpoint is used. Also, merged.mg is a compressed file and is being treated as an rcl, so what it returns is wrong.

We need to rethink and refactor these endpoints:

• GET /groups/{group_id}/files/{file_name}/xml
• GET /groups/{group_id}/files/{file_name}/json

Since we are not transforming files to XML in the current GET /groups/{group_id}/files/{file_name}/xml, the endpoints should be merged into one. In this new endpoint, it should be possible to set whether the result is json format or plain. The merged.mg should only be readable in plain format.

[IOTech17]

Great news, so the file is fine it is just the validator trying to validate a file that is being parsed as Json not XML.

Quick question, is there a guide to get all these file to be loaded for the non default groups?

[Selutario]

@IOTech17 agents can belong to multiple groups and will get files from all of them. Additionally, all of them belong to the default group unless explicitly indicated otherwise. Therefore, the easiest way to ensure that all agents have these files regardless of what other groups they belong to is to do nothing. You can take a look at this related documentation:

• Agent groups - Wazuh doc

The only alternative I can think of is to manually add the files to the other groups.

[GGP1]

Update

I began analysing the affected endpoints, the potential solutions and their implications.

[GGP1]

Update

I merged both endpoints into one that contains the return_format parameter which can be either plain (default) or json.

I decided to make plain the default format since most of the files inside the /var/ossec/etc/shared/<group> folder are .txt. I also opted to let users get the merged.mg file in json format if they prefer so (I couldn't think of a reason why it would be appropiate to return an error in such case).

[GGP1]

Update

• Replaced the return_format parameter with raw.
• Created a function that parses the merged.mg file converting it in a list of dictionaries with the fields file_name, file_size and file_content.
• Tested the changes manually and updated unit and integration tests.

[GGP1]

Update

Updated the raw default value to False instead of None and updated the base branch.

[GGP1]

Update

Applied the changes requested and tested them.

[Selutario]

Reopening this since it was closed automatically when merging the PR. The documentation will need to be updated (#21653 (review))

[GGP1]

Update

Updated the documentation RBAC reference.