-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Local DNS issue #763
Comments
Can you share a way to reproduce this? Also please share the output of the following commands when it's in a broken state: kubectl describe -n kube-system svc/kube-dns
kubectl logs -n kube-system svc/kube-dns
# run this and find the name of the coredns pod, e.g. "coredns-687f7d69ff-qvwph"
kubectl get pod -A
# substitute here
kubectl describe -n kube-system pod/coredns-XYZ
kubectl logs -n kube-system pod/coredns-XYZ |
You can use fluxcd to reproduce the error. Here are the command output ❯ kubectl describe -n kube-system svc/kube-dns
Name: kube-dns
Namespace: kube-system
Labels: k8s-app=kube-dns
kubernetes.io/cluster-service=true
kubernetes.io/name=CoreDNS
objectset.rio.cattle.io/hash=90462e4597bdcacaa7038840f786450ac707bfa2
Annotations: objectset.rio.cattle.io/applied:
H4sIAAAAAAAA/4yRQY/TMBCF/wqasxOSJm1SSxzQ7gUhoZUWuCAOE2eWmiS25ZkWoSr/HbnNikJV2JutefPpvTdHwGA/U2TrHWg4lKBgsK4HDY8UD9YQKJhIsEdB0EdA57ygWO84fX...
objectset.rio.cattle.io/id:
objectset.rio.cattle.io/owner-gvk: k3s.cattle.io/v1, Kind=Addon
objectset.rio.cattle.io/owner-name: orb-coredns
objectset.rio.cattle.io/owner-namespace: kube-system
prometheus.io/port: 9153
prometheus.io/scrape: true
Selector: k8s-app=kube-dns
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 192.168.194.138
IPs: 192.168.194.138
Port: dns 53/UDP
TargetPort: 53/UDP
Endpoints: 192.168.194.5:53
Port: dns-tcp 53/TCP
TargetPort: 53/TCP
Endpoints: 192.168.194.5:53
Port: metrics 9153/TCP
TargetPort: 9153/TCP
Endpoints: 192.168.194.5:9153
Session Affinity: None
Events: <none> ❯ kubectl describe -n kube-system pod/coredns-687f7d69ff-tmscz
Name: coredns-687f7d69ff-tmscz
Namespace: kube-system
Priority: 2000000000
Priority Class Name: system-cluster-critical
Service Account: coredns
Node: orbstack/198.19.249.2
Start Time: Mon, 13 Nov 2023 15:46:47 +1100
Labels: k8s-app=kube-dns
pod-template-hash=687f7d69ff
Annotations: <none>
Status: Running
IP: 192.168.194.5
IPs:
IP: 192.168.194.5
IP: fd07:b51a:cc66:a::5
Controlled By: ReplicaSet/coredns-687f7d69ff
Containers:
coredns:
Container ID: docker://9ad490cd7e154ab0f1129b2b55177750bca5a7464409367eeeace2be881f613c
Image: rancher/mirrored-coredns-coredns:1.10.1
Image ID: docker-pullable://rancher/mirrored-coredns-coredns@sha256:a11fafae1f8037cbbd66c5afa40ba2423936b72b4fd50a7034a7e8b955163594
Ports: 53/UDP, 53/TCP, 9153/TCP
Host Ports: 0/UDP, 0/TCP, 0/TCP
Args:
-conf
/etc/coredns/Corefile
State: Running
Started: Mon, 13 Nov 2023 15:46:48 +1100
Ready: True
Restart Count: 0
Limits:
memory: 170Mi
Requests:
cpu: 100m
memory: 70Mi
Readiness: http-get http://:8181/ready delay=0s timeout=1s period=2s #success=1 #failure=3
Environment: <none>
Mounts:
/etc/coredns from config-volume (ro)
/etc/coredns/custom from custom-config-volume (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-w4qtf (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: coredns
Optional: false
custom-config-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: coredns-custom
Optional: true
kube-api-access-w4qtf:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Tolerations: CriticalAddonsOnly op=Exists
node-role.kubernetes.io/control-plane:NoSchedule op=Exists
node-role.kubernetes.io/master:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Topology Spread Constraints: kubernetes.io/hostname:DoNotSchedule when max skew 1 is exceeded for selector k8s-app=kube-dns
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m11s default-scheduler Successfully assigned kube-system/coredns-687f7d69ff-tmscz to orbstack
Normal Pulled 2m11s kubelet Container image "rancher/mirrored-coredns-coredns:1.10.1" already present on machine
Normal Created 2m11s kubelet Created container coredns
Normal Started 2m11s kubelet Started container coredns ❯ kubectl logs -n kube-system pod/coredns-687f7d69ff-tmscz
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
.:53
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
[INFO] plugin/reload: Running configuration SHA512 = 2bf9b838333e20143c631516d1a474474b5ddb089d94e66f7831747dc364e3b7a92a481402ecd77264fd9b27c6abeea0e080f04d6bf43e10148b366e3bdd0e19
CoreDNS-1.10.1
linux/arm64, go1.20, 055b2c3
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.override
[WARNING] No files matching import glob pattern: /etc/coredns/custom/*.server |
@ylbeethoven Are these logs from when the issue is actively occurring, i.e. while you're getting Also, please provide exact steps to reproduce the issue with fluxcd. |
Yes
The error shows on GitRepository object. The steps are pretty simple if you have the repo set up... Bascially fluxCD I am sorry I can't put a deployment key on this public space but I am happy to show you a demo. I will DM you on mastadon if you are keen. |
Can you try restarting OrbStack, immediately running apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
labels:
k8s-app: kube-dns
k8slens-edit-resource-version: v1
kubernetes.io/name: CoreDNS
name: coredns
namespace: kube-system
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 0
selector:
matchLabels:
k8s-app: kube-dns
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
k8s-app: kube-dns
spec:
containers:
- args:
- -conf
- /etc/coredns/Corefile
image: rancher/mirrored-coredns-coredns:1.10.1
imagePullPolicy: IfNotPresent
name: coredns
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- containerPort: 9153
name: metrics
protocol: TCP
resources:
limits:
memory: 340Mi
requests:
cpu: 100m
memory: 70Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
add:
- NET_BIND_SERVICE
drop:
- all
readOnlyRootFilesystem: true
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/coredns
name: config-volume
readOnly: true
- mountPath: /etc/coredns/custom
name: custom-config-volume
readOnly: true
dnsPolicy: Default
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: coredns
serviceAccountName: coredns
terminationGracePeriodSeconds: 30
tolerations:
- key: CriticalAddonsOnly
operator: Exists
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
operator: Exists
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
topologySpreadConstraints:
- labelSelector:
matchLabels:
k8s-app: kube-dns
maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
volumes:
- configMap:
defaultMode: 420
items:
- key: Corefile
path: Corefile
- key: NodeHosts
path: NodeHosts
name: coredns
name: config-volume
- configMap:
defaultMode: 420
name: coredns-custom
optional: true
name: custom-config-volume
|
emm... this is weird. This fix works only once. (first try) After deleting and starting k8s, it did not work again. |
ok, I think I understand how to reproduce it. The fix seems to ONLY work the first time orbstack starts. In order to make it work, I need to
The fix does not work when I use |
Thanks for testing! Fixed for the next version. |
Released in v1.2.0 Canary 1. |
Released in v1.2.0. |
Describe the bug
Several pods/services are failing when running locally. I receive the following error:
failed to checkout and determine revision: unable to clone 'https://repo1.dso.mil/big-bang/bigbang.git': Get "https://repo1.dso.mil/big-bang/bigbang.git/info/refs?service=git-upload-pack": dial tcp: lookup repo1.dso.mil on 192.168.194.138:53: read udp 192.168.194.4:55533->192.168.194.138:53: read: connection refused
The git repo is available, but the local UDP traffic is being refused.
To Reproduce
No response
Expected behavior
No response
Diagnostic report (required)
OrbStack info:
Version: 1.0.1
Commit: 0d4cdcf185489f992cf8e3187884f11f4feab1aa (v1.0.1)
System info:
macOS: 14.0 (23A344)
CPU: arm64, 10 cores
CPU model: Apple M1 Pro
Model: MacBookPro18,1
Memory: 32 GiB
Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2023-10-31T13-23-46.810846Z.zip
Screenshots and additional context (optional)
No response
The text was updated successfully, but these errors were encountered: