DNS resolution fails in oc 3.11 cluster up on a Fedora 30 VM

[accorvin]

After deploying an openshift cluster (using oc cluster up --public-hostname=$IP_ADDRESS) I'm finding that pods/builds are unable to access external hosts (e.g. github.com). I first discovered this while trying to trigger a build (I got a could not resolve host error while trying to clone the source for the build). This is on a Fedora 30 VM running in OpenStack. I suspect that running in OpenStack may be causing the issue, although from what I can tell I've made the OpenStack network security group as permissive as possible.

Version

$ oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://10.0.154.211:8443
kubernetes v1.11.0+d4cacc0

Steps To Reproduce

1. Deploy an OpenShift cluster using oc cluster up (I specify my public hostname to be my VM's IP address
2. Run oc adm diagnostics diagnosticpod (see below output)

Current Result

Diagnostics fail. This results in, among other issues, image builds not being able to fetch source from external hosts.

Expected Result

Diagnostics should pass.

Additional Information

$ oc adm diagnostics diagnosticpod
[Note] Determining if client configuration exists for client/cluster diagnostics
Info:  Successfully read a client config file at '/home/fedora/.kube/config'

[Note] Running diagnostic: DiagnosticPod
       Description: Create a pod to run diagnostics from the application standpoint
       
ERROR: [DCli2012 from diagnostic DiagnosticPod@openshift/origin/pkg/oc/cli/admin/diagnostics/diagnostics/client/pod/run_diagnostics_pod.go:208]
       See the errors below in the output from the diagnostic pod:
       [Note] Running diagnostic: PodCheckAuth
              Description: Check that service account credentials authenticate as expected
              
       Info:  Service account token successfully authenticated to master
       ERROR: [DP1014 from diagnostic PodCheckAuth@openshift/origin/pkg/oc/cli/admin/diagnostics/diagnostics/client/pod/in_pod/auth.go:172]
              Request to integrated registry timed out; this typically indicates network or SDN problems.
              
       [Note] Running diagnostic: PodCheckDns
              Description: Check that DNS within a pod works as expected
              
       WARN:  [DP2014 from diagnostic PodCheckDns@openshift/origin/pkg/oc/cli/admin/diagnostics/diagnostics/client/pod/in_pod/dns.go:145]
              A request to the nameserver [172.30.0.2] timed out.
              This could be temporary but could also indicate network or DNS problems.
              
       [Note] Summary of diagnostics execution (version v3.11.0+3b2d3b6-227):
       [Note] Warnings seen: 1
       [Note] Errors seen: 1
       
[Note] Summary of diagnostics execution (version v3.11.0+0cbc58b):
[Note] Errors seen: 1```

[magick93]

Try going into your openshift folder (created in the directory you ran oc cluster up in) and edit the kubedns/resolv.conf, set nameserver to 8.8.8.8.

Then up the cluster.

[accorvin]

Nope, no luck. Here's my resolv.conf file:

[fedora@acorvin-workstation ~]$ cat openshift.local.clusterup/kubedns/resolv.conf
# Generated by NetworkManager
search openstacklocal
nameserver 8.8.8.8
nameserver 10.11.142.1
nameserver 10.11.5.19

To apply the change, I first started OpenShift (by running oc cluster up --public-hostname=10.0.154.53), then stopped the cluster, then applied the above change, than reran the cluster up command.

[jorge-romero]

Did hoy have any luck with this problem? I have the same problem running it on vmware in my w10 desktop.

[accorvin]

Nope, I have not solved this yet.

[86rishab]

I am facing same issue. Did you guys manage to resolve it?

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@charithjayasanka: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.