MongoDB metricsets incorrectly handling multiple hosts

[jsoriano]

Metricsets of the MongoDB module are doing their own parsing of the hosts setting (here), and using the result to configure the MongoDB client (here).

Usually, hosts parsing is done by the module, and individual metricsets should use the host data in the base metric set. This was also being done in this module at least till 8.2 (here).

The new approach produces failures when multiple hosts are used:

fetching data for metricset mongodb.dbstats: could not create mongodb client: could not create mongodb client: a direct connection cannot be made if multiple hosts are specified

For confirmed bugs, please report:

• Version: 8.3
• Operating System: Probably all.
• Discuss Forum URL: https://discuss.elastic.co/t/failed-to-monitor-mongo-cluster/308453
• Steps to Reproduce: Use the MongoDB Metricbeat module with multiple hosts.

[ritalwar]

The issue is fixed in PR : #34624.

Fix Details:

• Due to recent driver changes for MongoDB, a new approach is required to establish connections with the database.

• The previous method of using ClientOptions to set hosts and make direct connections needs to be updated.

• The new approach involves utilizing the ApplyURI method to configure the connection options based on the provided URI.

• For a single host, the direct connection can be specified in the URI itself, eliminating the need to explicitly set the option using ClientOptions.

• Therefore, as part of the fix, the step of setting the direct connection option using ClientOptions is removed.

• Additionally, the previous implementation involved setting hosts using the SetHosts option. This step is no longer necessary with the new approach. As, the ApplyURI option automatically handles the configuration based on the hosts specified in the URI.

[ritalwar]

During testing, some issues were identified with the fix. As a result, I have made modifications the fix. Currently, the fix is undergoing testing and review.

[rameshelastic]

Can we elaborate the issues identified @ritalwar ?

[ritalwar]

Findings:

1. During testing, it was discovered that the MongoDB connection was reverting to the default host and port, despite specifying a different host and port in the configuration. Further debugging revealed that the issue was caused by initializing the client options twice, resulting in the incorrect URI being applied.

2. Upon additional review, it was also observed that the MongoDB module requires code restructuring.

Options Considered:

• Fixing the issue and performing the code refactoring in the same pull request.

• Addressing the connection issue in this PR and scheduling the code refactoring as a separate pull request to align it with other Beats modules.

Proposed Actions:

• Validate the fix in the current PR to address the linked issue promptly.

• Initiate a separate PR for the module refactoring by filing a new issue. This refactoring will improve code organization and potentially enhance performance. We will provide the issue link for reference.

Status:
A fix has been implemented in this pull request, and it is currently undergoing testing.

[ritalwar]

Please refer the link to the refactoring issue: #35502