Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail to connect to Mongos on Sharded cluster in K8s #78

Closed
gsanchezgavier opened this issue Feb 11, 2022 · 2 comments
Closed

Fail to connect to Mongos on Sharded cluster in K8s #78

gsanchezgavier opened this issue Feb 11, 2022 · 2 comments
Labels
bug Categorizes issue or PR as related to a bug.

Comments

@gsanchezgavier
Copy link
Contributor

When monitoring a sharded clusters in K8s the integration fails to discover the mongos endpoint and no MongosSample is generated.

Description

Mongos metrics collection endpoints are discovered in the integration but in K8s the Id on collections.config.mongos contains the Pod name which cannot be resolved by the integration.

Expected Behavior

Support native mongos autodiscovery for K8s environments

Steps to Reproduce

Using bitnami mongo-sharded chart and configured the integration with discovery to the mongos pod

nri-mongodb:
        discovery:
          command:
            exec: /var/db/newrelic-infra/nri-discovery-kubernetes --port 10250 --tls
            match:
              label.app.kubernetes.io/component: mongos
        integrations:
          - name: nri-mongodb
            env:
              HOST: ${discovery.ip}
              ...

The integrations prints the error like:
[ERR] Failed to connect to mongos server <mongos-pod-name>:27017: no reachable servers

Your Environment

Additional context

I workaround this by using creating a service in the Agent namespace that points to the mongos instance and which have the same name of the mongos pod (using StatefulSet kind to deploy mongos to have predictable pod names). But this was just for the propose of validate the issue.

@gsanchezgavier gsanchezgavier added the bug Categorizes issue or PR as related to a bug. label Feb 11, 2022
@roobre
Copy link
Contributor

roobre commented Feb 11, 2022

Bitnami chart (and many others that deploy statefulSets) has a flag to deploy a service per replica, that could help with the DNS problem.

While there is currently no way to perform discovery on services, I think the name for this service should be very similar to the pod name (assuming it is deployed as a statefulSet). Perhaps we could use this to work around the problem.

@mangulonr
Copy link
Contributor

Hi

The new MongoDB OHI has been released in public preview, this version will be GA during June.

Some of the new features provided:

For more information check the following resources.

Because it is based on Prometheus the metric names are different from our legacy OHI, check our migration guide if you need more details.
Our legacy MongoDB OHI will be deprecated on FY24H2 (final date pending to be decided).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

4 participants