Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JIVA : Kubernetes - openebs - CrashLoopBackOff and I don't know why #3653

Open
RiioDeezy opened this issue Aug 27, 2023 · 4 comments
Open

JIVA : Kubernetes - openebs - CrashLoopBackOff and I don't know why #3653

RiioDeezy opened this issue Aug 27, 2023 · 4 comments
Labels
CNCF Community Community Reported Issue Need INFO Need info from the user No Activity - CLOSE issue Will CLOSE in 7 days

Comments

@RiioDeezy
Copy link

Description

Hello, I have a proxmox server with vm debian 11 I'm under the kubernetes environment installed on my vm. I try to install and configure the storage under openebs but it does not work. CrashLoopBackOff.

Your Environment

  • kubectl get nodes:

  • NAME STATUS ROLES AGE VERSION
    k8smaster1.utec.net Ready control-plane 13d v1.27.4
    k8smaster2.utec.lan Ready control-plane 13d v1.27.4
    k8smaster3.utec.lan Ready control-plane 13d v1.27.4
    k8sworker1.utec.net Ready 173m v1.28.0
    k8sworker2.utec.net Ready 3d10h v1.2

  • kubectl get services:

  • NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    kubernetes ClusterIP 10.96.0.1 443/TCP 13d

  • kubectl get sc:
    NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
    cstor-csi-disk cstor.csi.openebs.io Delete Immediate true 47h
    openebs-device openebs.io/local Delete WaitForFirstConsumer false 2d3h
    openebs-hostpath openebs.io/local Delete WaitForFirstConsumer false 2d3h
    openebs-jiva-csi-sc jiva.csi.openebs.io Delete Immediate true 2d

  • kubectl get pv:

  • No resources found

  • kubectl get pvc:

  • No resources found

  • OS (from /etc/os-release):

  • PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
    NAME="Debian GNU/Linux"
    VERSION_ID="11"
    VERSION="11 (bullseye)"
    VERSION_CODENAME=bullseye
    ID=debian
    HOME_URL="https://www.debian.org/"
    SUPPORT_URL="https://www.debian.org/support"
    BUG_REPORT_URL="https://bugs.debian.org/"

  • Kernel (from uname -a):

  • Linux k8smaster1.utec.net 5.10.0-23-amd64 Do we open issues here? Because easy. #1 SMP Debian 5.10.179-2 (2023-07-14) x86_64 GNU/Linux

  • Others:

  • kubectl get pods -n openebs
    NAME READY STATUS RESTARTS AGE
    cspc-operator-86ddc9f5dc-7gd8t 0/1 CrashLoopBackOff 24 (90s ago) 144m
    cvc-operator-59f746464f-n5z76 0/1 CrashLoopBackOff 257 (7s ago) 23h
    openebs-cstor-admission-server-5864dcb97f-tn8wm 0/1 CrashLoopBackOff 255 (69s ago) 23h
    openebs-cstor-csi-controller-0 0/6 CrashLoopBackOff 1746 (118s ago) 24h
    openebs-cstor-csi-node-7vt2p 2/2 Running 968 (44s ago) 47h
    openebs-cstor-csi-node-wtcp2 0/2 CrashLoopBackOff 70 (2m30s ago) 3h1m
    openebs-localpv-provisioner-7d6ccb7795-lxxw6 1/1 Running 261 (5m44s ago) 23h
    openebs-ndm-bj2l8 0/1 CrashLoopBackOff 500 (2m18s ago) 47h
    openebs-ndm-cluster-exporter-744779845d-jtjrg 0/1 CrashLoopBackOff 276 (4m47s ago) 23h
    openebs-ndm-dj5b8 0/1 CrashLoopBackOff 35 (2m42s ago) 3h1m
    openebs-ndm-node-exporter-wqsvb 0/1 CrashLoopBackOff 31 (2m39s ago) 3h1m
    openebs-ndm-node-exporter-zrxt6 0/1 CrashLoopBackOff 410 (2m7s ago) 47h
    openebs-ndm-operator-7fc69dbd55-sr9ht 0/1 CrashLoopBackOff 281 (4m12s ago) 23h

root@k8smaster1:/etc/kubernetes# kubectl get events -n openebs --sort-by=.metadata.creationTimestamp
LAST SEEN TYPE REASON OBJECT MESSAGE
7m9s Normal Killing pod/openebs-localpv-provisioner-7d6ccb7795-lxxw6 Stopping container openebs-provisioner-hostpath
15m Normal Killing pod/openebs-ndm-bj2l8 Stopping container node-disk-manager
2m Normal Killing pod/openebs-ndm-cluster-exporter-744779845d-jtjrg Stopping container ndm-cluster-exporter
107s Normal Killing pod/cvc-operator-59f746464f-n5z76 Stopping container cvc-operator
15m Normal Killing pod/openebs-cstor-admission-server-5864dcb97f-tn8wm Stopping container admission-webhook
16m Normal Killing pod/openebs-ndm-node-exporter-zrxt6 Stopping container node-disk-exporter
17m Warning BackOff pod/openebs-cstor-csi-node-7vt2p Back-off restarting failed container csi-node-driver-registrar in pod openebs-cstor-csi-node-7vt2p_openebs(58f93e1f-4363-429e-84c0-93fa4900e848)
2m32s Warning BackOff pod/openebs-ndm-bj2l8 Back-off restarting failed container node-disk-manager in pod openebs-ndm-bj2l8_openebs(06f74f7e-34fe-4cf5-8ef2-79c1e0f0bc8b)
2m28s Warning BackOff pod/openebs-ndm-operator-7fc69dbd55-sr9ht Back-off restarting failed container node-disk-operator in pod openebs-ndm-operator-7fc69dbd55-sr9ht_openebs(20a483d7-b081-46a2-9f2b-7e0598ca9769)
2m26s Warning BackOff pod/openebs-cstor-admission-server-5864dcb97f-tn8wm Back-off restarting failed container admission-webhook in pod openebs-cstor-admission-server-5864dcb97f-tn8wm_openebs(809f27b3-caeb-47e8-9e6a-3d3ea489234f)
2m34s Warning BackOff pod/openebs-ndm-node-exporter-zrxt6 Back-off restarting failed container node-disk-exporter in pod openebs-ndm-node-exporter-zrxt6_openebs(d268b6b6-f411-4a01-ba93-62cacd420631)
7m31s Warning BackOff pod/openebs-ndm-cluster-exporter-744779845d-jtjrg Back-off restarting failed container ndm-cluster-exporter in pod openebs-ndm-cluster-exporter-744779845d-jtjrg_openebs(d7923ede-114b-4f0f-b071-5f32df18e986)
2m32s Warning BackOff pod/openebs-localpv-provisioner-7d6ccb7795-lxxw6 Back-off restarting failed container openebs-provisioner-hostpath in pod openebs-localpv-provisioner-7d6ccb7795-lxxw6_openebs(b8bb0fcf-56fe-4d46-b975-1c2d5392c5af)
2m26s Warning BackOff pod/openebs-cstor-csi-controller-0 Back-off restarting failed container csi-resizer in pod openebs-cstor-csi-controller-0_openebs(8f1e5fd7-23ab-417d-929c-8685bb6aefc8)
62s Normal Killing pod/openebs-cstor-csi-node-7vt2p Stopping container csi-node-driver-registrar
7m31s Warning BackOff pod/cvc-operator-59f746464f-n5z76 Back-off restarting failed container cvc-operator in pod cvc-operator-59f746464f-n5z76_openebs(83f26f68-3787-46f0-98c8-4e1c85ad8928)
12m Normal Killing pod/openebs-cstor-csi-controller-0 Stopping container csi-resizer
11m Normal Killing pod/openebs-ndm-operator-7fc69dbd55-sr9ht Stopping container node-disk-operator
14m Normal Killing pod/openebs-ndm-dj5b8 Stopping container node-disk-manager
65s Normal Killing pod/openebs-cstor-csi-node-wtcp2 Stopping container csi-node-driver-registrar
4m16s Warning BackOff pod/openebs-ndm-dj5b8 Back-off restarting failed container node-disk-manager in pod openebs-ndm-dj5b8_openebs(5a373bb4-3e48-42df-92b5-c7fd38a1b3ac)
9m9s Warning BackOff pod/openebs-cstor-csi-node-wtcp2 Back-off restarting failed container csi-node-driver-registrar in pod openebs-cstor-csi-node-wtcp2_openebs(b72b37f3-c841-47fa-9b38-9f934989120f)
3m43s Normal Killing pod/openebs-ndm-node-exporter-wqsvb Stopping container node-disk-exporter
8m34s Warning BackOff pod/openebs-ndm-node-exporter-wqsvb Back-off restarting failed container node-disk-exporter in pod openebs-ndm-node-exporter-wqsvb_openebs(c0c91043-53a2-4328-96be-6f683b2da351)
9m12s Normal Killing pod/cspc-operator-86ddc9f5dc-7gd8t Stopping container cspc-operator
114s Warning BackOff pod/cspc-operator-86ddc9f5dc-7gd8t Back-off restarting failed container cspc-operator in pod cspc-operator-86ddc9f5dc-7gd8t_openebs(548ef5e2-099f-401f-aabf-b873a5c7b682)
39m Normal SandboxChanged pod/openebs-ndm-bj2l8 Pod sandbox changed, it will be killed and re-created.
34m Normal Pulled pod/openebs-cstor-csi-node-wtcp2 Container image "registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.8.0" already present on machine
33m Normal SandboxChanged pod/cvc-operator-59f746464f-n5z76 Pod sandbox changed, it will be killed and re-created.
26m Normal Pulled pod/openebs-ndm-dj5b8 Container image "openebs/node-disk-manager:2.1.0" already present on machine
60m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
58m Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_093bb5ac-798b-491f-9a3a-f983ce571230 became leader
53m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
52m Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_50f87239-8407-4d00-9706-4ccf26b707f7 became leader
47m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
45m Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_5a21fa63-c14a-47d0-85cb-b7519aed5150 became leader
41m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
41m Normal Pulled pod/cspc-operator-86ddc9f5dc-7gd8t Container image "openebs/cspc-operator:3.5.0" already present on machine
39m Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_4b4c5269-111c-4dcb-808e-806cee4b9e7d became leader
32m Normal Pulled pod/openebs-ndm-node-exporter-zrxt6 Container image "openebs/node-disk-exporter:2.1.0" already present on machine
27m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
26m Normal SandboxChanged pod/openebs-ndm-dj5b8 Pod sandbox changed, it will be killed and re-created.
26m Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_b6e7cba5-9818-4ec7-b60b-cffe0d434c10 became leader
21m Normal Killing pod/openebs-cstor-csi-controller-0 Stopping container csi-provisioner
17m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
17m Normal Pulled pod/openebs-cstor-csi-controller-0 Container image "registry.k8s.io/sig-storage/csi-resizer:v1.8.0" already present on machine
17m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
16m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
16m Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader
15m Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_496907fa-ae68-4976-af8c-6efe3161a8af became leader
8m1s Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_f7320850-cfac-4e5e-bf40-0a080821fb62 became leader
110s Normal LeaderElection endpoints/openebs.io-local openebs-localpv-provisioner-7d6ccb7795-lxxw6_73ba913f-ec98-4f7d-9b55-3e050e3753c3 became leader
106s Normal LeaderElection lease/external-resizer-cstor-csi-openebs-io openebs-cstor-csi-controller-0 became leader

root@k8smaster1:/etc/kubernetes# kubectl logs cspc-operator-86ddc9f5dc-7gd8t -n openebs --previous
I0827 21:03:39.235851 1 runner.go:40] Starting CSPC controller
I0827 21:03:39.235909 1 runner.go:43] Waiting for informer caches to sync
I0827 21:03:39.336476 1 runner.go:47] Starting CSPC workers
I0827 21:03:39.336522 1 runner.go:54] Started CSPC workers
root@k8smaster1:/etc/kubernetes# kubectl logs cvc-operator-59f746464f-n5z76 -n openebs --previous
I0827 21:06:01.997900 1 https.go:144] starting CVC REST server
I0827 21:06:01.999754 1 controller_base.go:244] Starting CstorVolumeConfig controller
I0827 21:06:02.000271 1 controller_base.go:247] Waiting for informer caches to sync
I0827 21:06:02.100772 1 controller_base.go:252] Starting CStorVolumeConfig workers
I0827 21:06:02.100850 1 controller_base.go:259] Started CStorVolumeConfig workers
root@k8smaster1:/etc/kubernetes# kubectl logs openebs-cstor-csi-controller-0 -n openebs --previous
Defaulted container "csi-resizer" out of: csi-resizer, csi-snapshotter, snapshot-controller, csi-provisioner, csi-attacher, cstor-csi-plugin
I0827 21:05:55.977749 1 main.go:93] Version : v1.8.0
I0827 21:05:55.977831 1 feature_gate.go:249] feature gates: &{map[]}
I0827 21:05:55.978586 1 connection.go:164] Connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock
I0827 21:05:56.979447 1 common.go:111] Probing CSI driver for readiness
I0827 21:05:56.979517 1 connection.go:193] GRPC call: /csi.v1.Identity/Probe
I0827 21:05:56.979552 1 connection.go:194] GRPC request: {}
I0827 21:05:56.982728 1 connection.go:200] GRPC response: {}
I0827 21:05:56.982746 1 connection.go:201] GRPC error:
I0827 21:05:56.982758 1 connection.go:193] GRPC call: /csi.v1.Identity/GetPluginInfo
I0827 21:05:56.982762 1 connection.go:194] GRPC request: {}
I0827 21:05:56.983190 1 connection.go:200] GRPC response: {"name":"cstor.csi.openebs.io","vendor_version":"3.5.0"}
I0827 21:05:56.983206 1 connection.go:201] GRPC error:
I0827 21:05:56.983214 1 main.go:141] CSI driver name: "cstor.csi.openebs.io"
I0827 21:05:56.983222 1 connection.go:193] GRPC call: /csi.v1.Identity/GetPluginCapabilities
I0827 21:05:56.983227 1 connection.go:194] GRPC request: {}
I0827 21:05:56.983772 1 connection.go:200] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]}
I0827 21:05:56.983789 1 connection.go:201] GRPC error:
I0827 21:05:56.983798 1 connection.go:193] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I0827 21:05:56.983817 1 connection.go:194] GRPC request: {}
I0827 21:05:56.984237 1 connection.go:200] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}}]}
I0827 21:05:56.984252 1 connection.go:201] GRPC error:
I0827 21:05:56.984260 1 connection.go:193] GRPC call: /csi.v1.Controller/ControllerGetCapabilities
I0827 21:05:56.984264 1 connection.go:194] GRPC request: {}
I0827 21:05:56.984553 1 connection.go:200] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":9}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":12}}},{"Type":{"Rpc":{"type":11}}}]}
I0827 21:05:56.984588 1 connection.go:201] GRPC error:
I0827 21:05:56.984739 1 controller.go:120] Register Pod informer for resizer cstor.csi.openebs.io
I0827 21:05:56.984980 1 leaderelection.go:245] attempting to acquire leader lease openebs/external-resizer-cstor-csi-openebs-io...
I0827 21:05:56.998706 1 leaderelection.go:255] successfully acquired lease openebs/external-resizer-cstor-csi-openebs-io
I0827 21:05:56.999034 1 leader_election.go:178] became leader, starting
I0827 21:05:56.999051 1 leader_election.go:185] new leader detected, current leader: openebs-cstor-csi-controller-0
I0827 21:05:56.999052 1 controller.go:255] Starting external resizer cstor.csi.openebs.io
I0827 21:05:56.999259 1 reflector.go:287] Starting reflector v1.PersistentVolumeClaim (10m0s) from k8s.io/client-go/informers/factory.go:150
I0827 21:05:56.999293 1 reflector.go:323] Listing and watching v1.PersistentVolumeClaim from k8s.io/client-go/informers/factory.go:150
I0827 21:05:56.999498 1 reflector.go:287] Starting reflector v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:150
I0827 21:05:56.999514 1 reflector.go:323] Listing and watching v1.PersistentVolume from k8s.io/client-go/informers/factory.go:150
I0827 21:05:56.999601 1 reflector.go:287] Starting reflector v1.Pod (10m0s) from k8s.io/client-go/informers/factory.go:150
I0827 21:05:56.999641 1 reflector.go:323] Listing and watching v1.Pod from k8s.io/client-go/informers/factory.go:150
I0827 21:05:57.013524 1 leaderelection.go:276] successfully renewed lease openebs/external-resizer-cstor-csi-openebs-io
I0827 21:05:57.099276 1 shared_informer.go:341] caches populated
root@k8smaster1:/etc/kubernetes# kubectl logs openebs-cstor-csi-node-7vt2p -n openebs --previous
Defaulted container "csi-node-driver-registrar" out of: csi-node-driver-registrar, cstor-csi-plugin
I0827 21:04:24.071258 1 main.go:167] Version: v2.8.0
I0827 21:04:24.071376 1 main.go:168] Running node-driver-registrar in mode=registration
I0827 21:04:24.072402 1 main.go:192] Attempting to open a gRPC connection with: "/plugin/csi.sock"
I0827 21:04:24.072439 1 connection.go:164] Connecting to unix:///plugin/csi.sock
root@k8smaster1:/etc/kubernetes# kubectl logs openebs-localpv-provisioner-7d6ccb7795-lxxw6 -n openebs --previous
I0827 21:00:31.993891 1 start.go:66] Starting Provisioner...
I0827 21:00:32.010021 1 start.go:127] Leader election enabled for localpv-provisioner
I0827 21:00:32.065975 1 leaderelection.go:248] attempting to acquire leader lease openebs/openebs.io-local...
I0827 21:00:49.479815 1 leaderelection.go:258] successfully acquired lease openebs/openebs.io-local
I0827 21:00:49.480003 1 event.go:282] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"openebs", Name:"openebs.io-local", UID:"47adcf5e-aabd-4d72-b200-7e2777b8207e", APIVersion:"v1", ResourceVersion:"3148398", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' openebs-localpv-provisioner-7d6ccb7795-lxxw6_73ba913f-ec98-4f7d-9b55-3e050e3753c3 became leader
I0827 21:00:49.480143 1 controller.go:810] Starting provisioner controller openebs.io/local_openebs-localpv-provisioner-7d6ccb7795-lxxw6_73ba913f-ec98-4f7d-9b55-3e050e3753c3!
I0827 21:00:49.580622 1 controller.go:859] Started provisioner controller openebs.io/local_openebs-localpv-provisioner-7d6ccb7795-lxxw6_73ba913f-ec98-4f7d-9b55-3e050e3753c3!
E0827 21:01:02.068029 1 googleanalytics.go:49] Post "https://www.google-analytics.com/collect": dial tcp: i/o timeout
root@k8smaster1:/etc/kubernetes# kubectl logs openebs-ndm-bj2l8 -n openebs --previous
[entrypoint.sh] launching ndm process.
E0827 21:04:19.012888 6 logs.go:35] unable to set flag, Error: no such flag -logtostderr
I0827 21:04:19.013276 6 commands.go:72] Starting Node Device Manager Daemon...
I0827 21:04:19.013388 6 commands.go:73] Version Tag : 2.1.0
I0827 21:04:19.013420 6 commands.go:74] GitCommit : b9f14b6af7a2f93ab915c43a4ebb402de427000b
I0827 21:04:19.013453 6 features.go:149] Feature gate: APIService, state: disable
I0827 21:04:19.013475 6 features.go:149] Feature gate: UseOSDisk, state: disable
I0827 21:04:19.013481 6 features.go:149] Feature gate: ChangeDetection, state: disable
I0827 21:04:19.013487 6 features.go:149] Feature gate: PartitionTableUUID, state: disable
I0827 21:04:19.013491 6 features.go:149] Feature gate: GPTBasedUUID, state: enable
I0827 21:04:19.114560 6 request.go:614] Waited for 88.926392ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/openebs.io/v1alpha1?timeout=32s
I0827 21:04:19.164390 6 request.go:614] Waited for 138.733773ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/longhorn.io/v1beta2?timeout=32s
I0827 21:04:19.214465 6 request.go:614] Waited for 188.842808ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/longhorn.io/v1beta1?timeout=32s
I0827 21:04:19.414398 6 request.go:614] Waited for 97.832058ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/api/v1?timeout=32s
I0827 21:04:19.464222 6 request.go:614] Waited for 147.651315ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/apiregistration.k8s.io/v1?timeout=32s
I0827 21:04:19.513796 6 request.go:614] Waited for 197.231647ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/coordination.k8s.io/v1?timeout=32s
I0827 21:04:19.564502 6 request.go:614] Waited for 247.934928ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/scheduling.k8s.io/v1?timeout=32s
I0827 21:04:19.613961 6 request.go:614] Waited for 297.378795ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/discovery.k8s.io/v1?timeout=32s
I0827 21:04:19.664354 6 request.go:614] Waited for 347.787126ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/node.k8s.io/v1?timeout=32s
I0827 21:04:19.714620 6 request.go:614] Waited for 398.009174ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta3?timeout=32s
I0827 21:04:19.764110 6 request.go:614] Waited for 447.44169ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/apps/v1?timeout=32s
I0827 21:04:19.814701 6 request.go:614] Waited for 498.098615ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta2?timeout=32s
I0827 21:04:19.864211 6 request.go:614] Waited for 547.588302ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/events.k8s.io/v1?timeout=32s
I0827 21:04:19.914792 6 request.go:614] Waited for 598.13291ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/acme.cert-manager.io/v1?timeout=32s
I0827 21:04:19.964152 6 request.go:614] Waited for 647.46264ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/cstor.openebs.io/v1?timeout=32s
I0827 21:04:20.014600 6 request.go:614] Waited for 697.901059ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/crd.projectcalico.org/v1?timeout=32s
I0827 21:04:20.064153 6 request.go:614] Waited for 747.417455ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/snapshot.storage.k8s.io/v1?timeout=32s
I0827 21:04:20.114773 6 request.go:614] Waited for 798.026909ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/authentication.k8s.io/v1?timeout=32s
I0827 21:04:20.164434 6 request.go:614] Waited for 847.703105ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/cert-manager.io/v1?timeout=32s
I0827 21:04:20.214183 6 request.go:614] Waited for 897.414699ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/policy/v1?timeout=32s
I0827 21:04:20.264764 6 request.go:614] Waited for 948.009233ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/rbac.authorization.k8s.io/v1?timeout=32s
I0827 21:04:20.314155 6 request.go:614] Waited for 997.391257ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/storage.k8s.io/v1?timeout=32s
I0827 21:04:20.364742 6 request.go:614] Waited for 1.047963927s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/admissionregistration.k8s.io/v1?timeout=32s
I0827 21:04:20.364781 6 request.go:682] Waited for 1.047963927s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/admissionregistration.k8s.io/v1?timeout=32s
I0827 21:04:20.414262 6 request.go:614] Waited for 1.097465157s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/apiextensions.k8s.io/v1?timeout=32s
I0827 21:04:20.464751 6 request.go:614] Waited for 1.147941808s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/autoscaling/v1?timeout=32s
I0827 21:04:20.514247 6 request.go:614] Waited for 1.197433409s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/openebs.io/v1alpha1?timeout=32s
I0827 21:04:20.564122 6 request.go:614] Waited for 1.247322404s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/authorization.k8s.io/v1?timeout=32s
I0827 21:04:20.614716 6 request.go:614] Waited for 1.297900913s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/longhorn.io/v1beta2?timeout=32s
I0827 21:04:20.664397 6 request.go:614] Waited for 1.347571852s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/autoscaling/v2?timeout=32s
I0827 21:04:20.714034 6 request.go:614] Waited for 1.397225904s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/longhorn.io/v1beta1?timeout=32s
I0827 21:04:20.764609 6 request.go:614] Waited for 1.447791575s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/batch/v1?timeout=32s
I0827 21:04:20.814018 6 request.go:614] Waited for 1.497201995s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/certificates.k8s.io/v1?timeout=32s
I0827 21:04:20.864692 6 request.go:614] Waited for 1.547804884s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/networking.k8s.io/v1?timeout=32s
I0827 21:04:20.914960 6 request.go:614] Waited for 1.59808197s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/volumesnapshot.external-storage.k8s.io/v1?timeout=32s
I0827 21:04:21.024563 6 request.go:614] Waited for 80.299153ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/volumesnapshot.external-storage.k8s.io/v1?timeout=32s
I0827 21:04:21.075059 6 request.go:614] Waited for 130.813ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/snapshot.storage.k8s.io/v1?timeout=32s
I0827 21:04:21.124354 6 request.go:614] Waited for 180.029619ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/openebs.io/v1alpha1?timeout=32s
I0827 21:04:21.324812 6 request.go:614] Waited for 98.708169ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/networking.k8s.io/v1?timeout=32s
I0827 21:04:21.375318 6 request.go:614] Waited for 149.176413ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/api/v1?timeout=32s
I0827 21:04:21.425236 6 request.go:614] Waited for 199.100286ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/apiregistration.k8s.io/v1?timeout=32s
I0827 21:04:21.475021 6 request.go:614] Waited for 248.888891ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/policy/v1?timeout=32s
I0827 21:04:21.524616 6 request.go:614] Waited for 298.475524ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/apps/v1?timeout=32s
I0827 21:04:21.574619 6 request.go:614] Waited for 348.440042ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/rbac.authorization.k8s.io/v1?timeout=32s
I0827 21:04:21.625203 6 request.go:614] Waited for 399.052461ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/events.k8s.io/v1?timeout=32s
I0827 21:04:21.674640 6 request.go:614] Waited for 448.4664ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/storage.k8s.io/v1?timeout=32s
I0827 21:04:21.725094 6 request.go:614] Waited for 498.929439ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/authentication.k8s.io/v1?timeout=32s
I0827 21:04:21.775171 6 request.go:614] Waited for 549.006041ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/admissionregistration.k8s.io/v1?timeout=32s
I0827 21:04:21.824609 6 request.go:614] Waited for 598.442824ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/authorization.k8s.io/v1?timeout=32s
I0827 21:04:21.874463 6 request.go:614] Waited for 648.29584ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/apiextensions.k8s.io/v1?timeout=32s
I0827 21:04:21.924342 6 request.go:614] Waited for 698.147883ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/autoscaling/v2?timeout=32s
I0827 21:04:21.974789 6 request.go:614] Waited for 748.600796ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/scheduling.k8s.io/v1?timeout=32s
I0827 21:04:22.025362 6 request.go:614] Waited for 799.182023ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/autoscaling/v1?timeout=32s
I0827 21:04:22.074727 6 request.go:614] Waited for 848.543776ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/coordination.k8s.io/v1?timeout=32s
I0827 21:04:22.125341 6 request.go:614] Waited for 899.153813ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/batch/v1?timeout=32s
I0827 21:04:22.175118 6 request.go:614] Waited for 948.927879ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/node.k8s.io/v1?timeout=32s
I0827 21:04:22.224669 6 request.go:614] Waited for 998.478346ms due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/certificates.k8s.io/v1?timeout=32s
I0827 21:04:22.275237 6 request.go:614] Waited for 1.049043113s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/discovery.k8s.io/v1?timeout=32s
I0827 21:04:22.275275 6 request.go:682] Waited for 1.049043113s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/discovery.k8s.io/v1?timeout=32s
I0827 21:04:22.324795 6 request.go:614] Waited for 1.098594289s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/cstor.openebs.io/v1?timeout=32s
I0827 21:04:22.375294 6 request.go:614] Waited for 1.149091278s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/snapshot.storage.k8s.io/v1?timeout=32s
I0827 21:04:22.424954 6 request.go:614] Waited for 1.198743397s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta3?timeout=32s
I0827 21:04:22.474681 6 request.go:614] Waited for 1.248463741s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/volumesnapshot.external-storage.k8s.io/v1?timeout=32s
I0827 21:04:22.524334 6 request.go:614] Waited for 1.298116689s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta2?timeout=32s
I0827 21:04:22.575188 6 request.go:614] Waited for 1.348955459s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/openebs.io/v1alpha1?timeout=32s
I0827 21:04:22.624764 6 request.go:614] Waited for 1.398539308s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/acme.cert-manager.io/v1?timeout=32s
I0827 21:04:22.675121 6 request.go:614] Waited for 1.44888318s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/longhorn.io/v1beta2?timeout=32s
I0827 21:04:22.724896 6 request.go:614] Waited for 1.498666771s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/cert-manager.io/v1?timeout=32s
I0827 21:04:22.774544 6 request.go:614] Waited for 1.548285861s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/longhorn.io/v1beta1?timeout=32s
I0827 21:04:22.825080 6 request.go:614] Waited for 1.598795778s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/crd.projectcalico.org/v1?timeout=32s
I0827 21:04:22.832174 6 controller.go:291] BlockDevice CRD is available
I0827 21:04:22.846028 6 filter.go:60] registering filters
I0827 21:04:22.846067 6 filter.go:68] configured os disk exclude filter : state enable
E0827 21:04:22.847193 6 osdiskexcludefilter.go:106] unable to configure os disk filter for mountpoint: /boot, error: could not get device mount attributes, Path/MountPoint not present in mounts file
I0827 21:04:22.847218 6 filter.go:68] configured vendor filter : state enable
I0827 21:04:22.847229 6 filter.go:68] configured path filter : state enable
I0827 21:04:22.847236 6 filter.go:68] configured device validity filter : state enable
I0827 21:04:22.847253 6 probe.go:66] registering probes
I0827 21:04:22.847265 6 probe.go:89] configured seachest probe : state disable
I0827 21:04:22.847272 6 probe.go:89] configured smart probe : state enable
I0827 21:04:22.847279 6 probe.go:89] configured mount probe : state enable
I0827 21:04:22.847297 6 probe.go:89] configured udev probe : state enable
I0827 21:04:22.848536 6 udevprobe.go:358] starting udev probe listener
I0827 21:04:22.848984 6 udevprobe.go:216] device: /dev/sr0, Serial: QM00003 filled during udev scan
I0827 21:04:22.849005 6 udevprobe.go:220] device: /dev/sr0, PartitionTableUUID: 6f65095c filled during udev scan
I0827 21:04:22.849011 6 udevprobe.go:228] device: /dev/sr0, FileSystemUUID: 2023-04-29-11-53-15-00 filled during udev scan
I0827 21:04:22.850098 6 udevprobe.go:245] Dependents of /dev/sr0 : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0827 21:04:22.850158 6 udevprobe.go:255] Device: /dev/sr0 is of type: disk
I0827 21:04:22.850397 6 udevprobe.go:216] device: /dev/sda, Serial: drive-scsi0 filled during udev scan
I0827 21:04:22.850406 6 udevprobe.go:220] device: /dev/sda, PartitionTableUUID: ec82e1a0 filled during udev scan
I0827 21:04:22.850730 6 udevprobe.go:245] Dependents of /dev/sda : {Parent: Partitions:[/dev/sda1 /dev/sda2 /dev/sda5] Holders:[] Slaves:[]}
I0827 21:04:22.850744 6 udevprobe.go:255] Device: /dev/sda is of type: disk
I0827 21:04:22.850977 6 udevprobe.go:216] device: /dev/sda1, Serial: drive-scsi0 filled during udev scan
I0827 21:04:22.850987 6 udevprobe.go:220] device: /dev/sda1, PartitionTableUUID: ec82e1a0 filled during udev scan
I0827 21:04:22.850993 6 udevprobe.go:224] device: /dev/sda1, PartitionEntryUUID: ec82e1a0-01 filled during udev scan
I0827 21:04:22.850998 6 udevprobe.go:228] device: /dev/sda1, FileSystemUUID: 01b09d11-3d90-475e-beaf-b69cd59e3dcd filled during udev scan
I0827 21:04:22.851214 6 udevprobe.go:245] Dependents of /dev/sda1 : {Parent:/dev/sda Partitions:[] Holders:[] Slaves:[]}
I0827 21:04:22.851524 6 udevprobe.go:255] Device: /dev/sda1 is of type: partition
I0827 21:04:22.851740 6 udevprobe.go:216] device: /dev/sda2, Serial: drive-scsi0 filled during udev scan
I0827 21:04:22.851748 6 udevprobe.go:220] device: /dev/sda2, PartitionTableUUID: ec82e1a0 filled during udev scan
I0827 21:04:22.851754 6 udevprobe.go:224] device: /dev/sda2, PartitionEntryUUID: ec82e1a0-02 filled during udev scan
I0827 21:04:22.851965 6 udevprobe.go:245] Dependents of /dev/sda2 : {Parent:/dev/sda Partitions:[] Holders:[] Slaves:[]}
I0827 21:04:22.851977 6 udevprobe.go:255] Device: /dev/sda2 is of type: partition
I0827 21:04:22.852186 6 udevprobe.go:216] device: /dev/sda5, Serial: drive-scsi0 filled during udev scan
I0827 21:04:22.852195 6 udevprobe.go:220] device: /dev/sda5, PartitionTableUUID: ec82e1a0 filled during udev scan
I0827 21:04:22.852200 6 udevprobe.go:224] device: /dev/sda5, PartitionEntryUUID: ec82e1a0-05 filled during udev scan
I0827 21:04:22.852205 6 udevprobe.go:228] device: /dev/sda5, FileSystemUUID: a9f26ca7-e8f1-4d9a-9a23-4b6aa3d07c0e filled during udev scan
I0827 21:04:22.852413 6 udevprobe.go:245] Dependents of /dev/sda5 : {Parent:/dev/sda Partitions:[] Holders:[] Slaves:[]}
I0827 21:04:22.852429 6 udevprobe.go:255] Device: /dev/sda5 is of type: partition
I0827 21:04:22.852615 6 udevprobe.go:216] device: /dev/sdb, Serial: drive-scsi1 filled during udev scan
I0827 21:04:22.852623 6 udevprobe.go:220] device: /dev/sdb, PartitionTableUUID: beb7b1e6-c8f7-402f-9620-e4cd340915a6 filled during udev scan
I0827 21:04:22.852936 6 udevprobe.go:245] Dependents of /dev/sdb : {Parent: Partitions:[/dev/sdb1] Holders:[] Slaves:[]}
I0827 21:04:22.852950 6 udevprobe.go:255] Device: /dev/sdb is of type: disk
I0827 21:04:22.853142 6 udevprobe.go:216] device: /dev/sdb1, Serial: drive-scsi1 filled during udev scan
I0827 21:04:22.853149 6 udevprobe.go:220] device: /dev/sdb1, PartitionTableUUID: beb7b1e6-c8f7-402f-9620-e4cd340915a6 filled during udev scan
I0827 21:04:22.853155 6 udevprobe.go:224] device: /dev/sdb1, PartitionEntryUUID: a7687955-8e3f-4357-a50e-95f2e35cb263 filled during udev scan
I0827 21:04:22.853366 6 udevprobe.go:245] Dependents of /dev/sdb1 : {Parent:/dev/sdb Partitions:[] Holders:[] Slaves:[]}
I0827 21:04:22.853378 6 udevprobe.go:255] Device: /dev/sdb1 is of type: partition
I0827 21:04:22.881609 6 blockdevicestore.go:131] eventcode=ndm.blockdevice.deactivate.success msg=Deactivated blockdevice rname=blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:04:22.881693 6 probe.go:89] configured sysfs probe : state enable
I0827 21:04:22.881719 6 probe.go:89] configured used-by probe : state enable
I0827 21:04:22.881740 6 probe.go:89] configured Custom Tag Probe : state enable
I0827 21:04:22.881760 6 probe.go:89] configured blkid probe : state enable
I0827 21:04:22.881801 6 sparsefilegenerator.go:145] No sparse file path/size provided. Skip creating sparse files.
I0827 21:04:22.882266 6 controller.go:321] started the controller
I0827 21:04:22.891141 6 eventhandler.go:69] Processing details for /dev/sr0
I0827 21:04:22.891577 6 udevprobe.go:294] device: /dev/sr0, Model: QEMU_DVD-ROM filled by udev probe
I0827 21:04:22.891615 6 udevprobe.go:302] device: /dev/sr0, Serial: QM00003 filled by udev probe
I0827 21:04:22.891635 6 udevprobe.go:310] device: /dev/sr0, IDType: cd filled by udev probe
I0827 21:04:22.891668 6 probe.go:119] details filled by udev probe
I0827 21:04:22.891785 6 sysfsprobe.go:97] blockdevice path: /dev/sr0 capacity :407896064 filled by sysfs probe.
I0827 21:04:22.891860 6 sysfsprobe.go:125] blockdevice path: /dev/sr0 logical block size :2048 filled by sysfs probe.
I0827 21:04:22.891918 6 sysfsprobe.go:137] blockdevice path: /dev/sr0 physical block size :2048 filled by sysfs probe.
I0827 21:04:22.891974 6 sysfsprobe.go:149] blockdevice path: /dev/sr0 hardware sector size :2048 filled by sysfs probe.
I0827 21:04:22.892074 6 sysfsprobe.go:160] blockdevice path: /dev/sr0 drive type :HDD filled by sysfs probe.
I0827 21:04:22.892111 6 probe.go:119] details filled by sysfs probe
E0827 21:04:22.892164 6 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0827 21:04:22.892199 6 probe.go:119] details filled by smart probe
I0827 21:04:22.892526 6 mountprobe.go:134] no mount point found for /dev/sr0. clearing mount points if any
I0827 21:04:22.892557 6 probe.go:119] details filled by mount probe
I0827 21:04:22.940200 6 probe.go:119] details filled by blkid probe
I0827 21:04:22.940224 6 usedbyprobe.go:122] device: /dev/sr0 is not having any zfs partitions
I0827 21:04:22.942114 6 probe.go:119] details filled by used-by probe
I0827 21:04:22.942306 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:22.942333 6 addhandler.go:53] device: /dev/sr0 does not exist in cache, the device is now connected to this node
I0827 21:04:22.942360 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sr0
I0827 21:04:22.942387 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sr0
I0827 21:04:22.942407 6 filter.go:89] /dev/sr0 ignored by path filter
I0827 21:04:22.942421 6 eventhandler.go:69] Processing details for /dev/sda
I0827 21:04:22.942667 6 udevprobe.go:294] device: /dev/sda, Model: QEMU_HARDDISK filled by udev probe
I0827 21:04:22.942694 6 udevprobe.go:302] device: /dev/sda, Serial: drive-scsi0 filled by udev probe
I0827 21:04:22.942700 6 udevprobe.go:306] device: /dev/sda, Vendor: QEMU filled by udev probe
I0827 21:04:22.942705 6 udevprobe.go:310] device: /dev/sda, IDType: disk filled by udev probe
I0827 21:04:22.942717 6 probe.go:119] details filled by udev probe
I0827 21:04:22.942833 6 sysfsprobe.go:97] blockdevice path: /dev/sda capacity :64424509440 filled by sysfs probe.
I0827 21:04:22.942882 6 sysfsprobe.go:125] blockdevice path: /dev/sda logical block size :512 filled by sysfs probe.
I0827 21:04:22.942955 6 sysfsprobe.go:137] blockdevice path: /dev/sda physical block size :512 filled by sysfs probe.
I0827 21:04:22.943003 6 sysfsprobe.go:149] blockdevice path: /dev/sda hardware sector size :512 filled by sysfs probe.
I0827 21:04:22.943049 6 sysfsprobe.go:160] blockdevice path: /dev/sda drive type :HDD filled by sysfs probe.
I0827 21:04:22.943064 6 probe.go:119] details filled by sysfs probe
I0827 21:04:22.943472 6 probe.go:119] details filled by smart probe
I0827 21:04:22.943804 6 mountprobe.go:134] no mount point found for /dev/sda. clearing mount points if any
I0827 21:04:22.943812 6 probe.go:119] details filled by mount probe
I0827 21:04:22.946078 6 probe.go:119] details filled by blkid probe
I0827 21:04:22.946099 6 usedbyprobe.go:122] device: /dev/sda is not having any zfs partitions
I0827 21:04:22.946157 6 probe.go:119] details filled by used-by probe
I0827 21:04:22.946163 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:22.946169 6 addhandler.go:53] device: /dev/sda does not exist in cache, the device is now connected to this node
I0827 21:04:22.946196 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sda
I0827 21:04:22.946227 6 filter.go:89] /dev/sda ignored by os disk exclude filter
I0827 21:04:22.946235 6 eventhandler.go:69] Processing details for /dev/sda1
I0827 21:04:22.946489 6 udevprobe.go:294] device: /dev/sda1, Model: QEMU_HARDDISK filled by udev probe
I0827 21:04:22.946497 6 udevprobe.go:302] device: /dev/sda1, Serial: drive-scsi0 filled by udev probe
I0827 21:04:22.946501 6 udevprobe.go:306] device: /dev/sda1, Vendor: QEMU filled by udev probe
I0827 21:04:22.946506 6 udevprobe.go:310] device: /dev/sda1, IDType: disk filled by udev probe
I0827 21:04:22.946519 6 probe.go:119] details filled by udev probe
I0827 21:04:22.946633 6 sysfsprobe.go:97] blockdevice path: /dev/sda1 capacity :63399002112 filled by sysfs probe.
I0827 21:04:22.946722 6 sysfsprobe.go:125] blockdevice path: /dev/sda1 logical block size :512 filled by sysfs probe.
I0827 21:04:22.946752 6 sysfsprobe.go:137] blockdevice path: /dev/sda1 physical block size :512 filled by sysfs probe.
I0827 21:04:22.946796 6 sysfsprobe.go:149] blockdevice path: /dev/sda1 hardware sector size :512 filled by sysfs probe.
I0827 21:04:22.946826 6 sysfsprobe.go:160] blockdevice path: /dev/sda1 drive type :HDD filled by sysfs probe.
I0827 21:04:22.946832 6 probe.go:119] details filled by sysfs probe
I0827 21:04:22.947188 6 probe.go:119] details filled by smart probe
I0827 21:04:22.947577 6 probe.go:119] details filled by mount probe
I0827 21:04:22.947591 6 probe.go:119] details filled by blkid probe
I0827 21:04:22.948996 6 probe.go:119] details filled by used-by probe
I0827 21:04:22.949125 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:22.949193 6 addhandler.go:53] device: /dev/sda1 does not exist in cache, the device is now connected to this node
I0827 21:04:22.949270 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sda1
I0827 21:04:22.949347 6 filter.go:89] /dev/sda1 ignored by os disk exclude filter
I0827 21:04:22.949404 6 eventhandler.go:69] Processing details for /dev/sda2
I0827 21:04:22.949690 6 udevprobe.go:294] device: /dev/sda2, Model: QEMU_HARDDISK filled by udev probe
I0827 21:04:22.949761 6 udevprobe.go:302] device: /dev/sda2, Serial: drive-scsi0 filled by udev probe
I0827 21:04:22.949815 6 udevprobe.go:306] device: /dev/sda2, Vendor: QEMU filled by udev probe
I0827 21:04:22.949883 6 udevprobe.go:310] device: /dev/sda2, IDType: disk filled by udev probe
I0827 21:04:22.949965 6 probe.go:119] details filled by udev probe
I0827 21:04:22.950129 6 sysfsprobe.go:97] blockdevice path: /dev/sda2 capacity :1024 filled by sysfs probe.
I0827 21:04:22.950260 6 sysfsprobe.go:125] blockdevice path: /dev/sda2 logical block size :512 filled by sysfs probe.
I0827 21:04:22.950339 6 sysfsprobe.go:137] blockdevice path: /dev/sda2 physical block size :512 filled by sysfs probe.
I0827 21:04:22.950428 6 sysfsprobe.go:149] blockdevice path: /dev/sda2 hardware sector size :512 filled by sysfs probe.
I0827 21:04:22.950491 6 sysfsprobe.go:160] blockdevice path: /dev/sda2 drive type :HDD filled by sysfs probe.
I0827 21:04:22.950504 6 probe.go:119] details filled by sysfs probe
I0827 21:04:22.950809 6 probe.go:119] details filled by smart probe
I0827 21:04:22.951198 6 mountprobe.go:134] no mount point found for /dev/sda2. clearing mount points if any
I0827 21:04:22.951272 6 probe.go:119] details filled by mount probe
I0827 21:04:22.952638 6 probe.go:119] details filled by blkid probe
E0827 21:04:22.956428 6 usedbyprobe.go:160] error reading spdk signature from device: /dev/sda2, error reading from /dev/sda2: unexpected EOF
I0827 21:04:22.956491 6 probe.go:119] details filled by used-by probe
I0827 21:04:22.956513 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:22.956533 6 addhandler.go:53] device: /dev/sda2 does not exist in cache, the device is now connected to this node
I0827 21:04:22.956571 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sda2
I0827 21:04:22.956609 6 filter.go:89] /dev/sda2 ignored by os disk exclude filter
I0827 21:04:22.956630 6 eventhandler.go:69] Processing details for /dev/sda5
I0827 21:04:22.956905 6 udevprobe.go:294] device: /dev/sda5, Model: QEMU_HARDDISK filled by udev probe
I0827 21:04:22.956936 6 udevprobe.go:302] device: /dev/sda5, Serial: drive-scsi0 filled by udev probe
I0827 21:04:22.956955 6 udevprobe.go:306] device: /dev/sda5, Vendor: QEMU filled by udev probe
I0827 21:04:22.956972 6 udevprobe.go:310] device: /dev/sda5, IDType: disk filled by udev probe
I0827 21:04:22.957003 6 probe.go:119] details filled by udev probe
I0827 21:04:22.957148 6 sysfsprobe.go:97] blockdevice path: /dev/sda5 capacity :1022361600 filled by sysfs probe.
I0827 21:04:22.957262 6 sysfsprobe.go:125] blockdevice path: /dev/sda5 logical block size :512 filled by sysfs probe.
I0827 21:04:22.957318 6 sysfsprobe.go:137] blockdevice path: /dev/sda5 physical block size :512 filled by sysfs probe.
I0827 21:04:22.957381 6 sysfsprobe.go:149] blockdevice path: /dev/sda5 hardware sector size :512 filled by sysfs probe.
I0827 21:04:22.957507 6 sysfsprobe.go:160] blockdevice path: /dev/sda5 drive type :HDD filled by sysfs probe.
I0827 21:04:22.957575 6 probe.go:119] details filled by sysfs probe
I0827 21:04:22.957868 6 probe.go:119] details filled by smart probe
I0827 21:04:22.958222 6 mountprobe.go:134] no mount point found for /dev/sda5. clearing mount points if any
I0827 21:04:22.958290 6 probe.go:119] details filled by mount probe
I0827 21:04:22.958346 6 probe.go:119] details filled by blkid probe
I0827 21:04:22.989200 6 probe.go:119] details filled by used-by probe
I0827 21:04:22.989333 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:22.989363 6 addhandler.go:53] device: /dev/sda5 does not exist in cache, the device is now connected to this node
I0827 21:04:22.989421 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sda5
I0827 21:04:22.989477 6 filter.go:89] /dev/sda5 ignored by os disk exclude filter
I0827 21:04:22.989504 6 eventhandler.go:69] Processing details for /dev/sdb
I0827 21:04:22.989755 6 udevprobe.go:294] device: /dev/sdb, Model: QEMU_HARDDISK filled by udev probe
I0827 21:04:22.989783 6 udevprobe.go:302] device: /dev/sdb, Serial: drive-scsi1 filled by udev probe
I0827 21:04:22.989818 6 udevprobe.go:306] device: /dev/sdb, Vendor: QEMU filled by udev probe
I0827 21:04:22.989836 6 udevprobe.go:310] device: /dev/sdb, IDType: disk filled by udev probe
I0827 21:04:22.989875 6 probe.go:119] details filled by udev probe
I0827 21:04:22.990030 6 sysfsprobe.go:97] blockdevice path: /dev/sdb capacity :26843545600 filled by sysfs probe.
I0827 21:04:22.990105 6 sysfsprobe.go:125] blockdevice path: /dev/sdb logical block size :512 filled by sysfs probe.
I0827 21:04:22.990181 6 sysfsprobe.go:137] blockdevice path: /dev/sdb physical block size :512 filled by sysfs probe.
I0827 21:04:22.990259 6 sysfsprobe.go:149] blockdevice path: /dev/sdb hardware sector size :512 filled by sysfs probe.
I0827 21:04:22.990339 6 sysfsprobe.go:160] blockdevice path: /dev/sdb drive type :HDD filled by sysfs probe.
I0827 21:04:22.990364 6 probe.go:119] details filled by sysfs probe
I0827 21:04:22.990696 6 probe.go:119] details filled by smart probe
I0827 21:04:22.991063 6 mountprobe.go:134] no mount point found for /dev/sdb. clearing mount points if any
I0827 21:04:22.991092 6 probe.go:119] details filled by mount probe
I0827 21:04:23.009877 6 probe.go:119] details filled by blkid probe
I0827 21:04:23.009917 6 usedbyprobe.go:122] device: /dev/sdb is not having any zfs partitions
I0827 21:04:23.010325 6 probe.go:119] details filled by used-by probe
I0827 21:04:23.011586 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:23.011614 6 addhandler.go:53] device: /dev/sdb does not exist in cache, the device is now connected to this node
I0827 21:04:23.011753 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sdb
I0827 21:04:23.011779 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sdb
I0827 21:04:23.011817 6 eventhandler.go:82] Processed details for /dev/sdb
I0827 21:04:23.011825 6 addhandler.go:106] checking if device: /dev/sdb can be uniquely identified
I0827 21:04:23.011832 6 addhandler.go:110] device: /dev/sdb cannot be uniquely identified
I0827 21:04:23.011839 6 addhandler.go:113] device: /dev/sdb has holders/partitions. {Parent: Partitions:[/dev/sdb1] Holders:[] Slaves:[]}
I0827 21:04:23.011850 6 eventhandler.go:69] Processing details for /dev/sdb1
I0827 21:04:23.012351 6 probe.go:119] details filled by udev probe
I0827 21:04:23.013119 6 sysfsprobe.go:97] blockdevice path: /dev/sdb1 capacity :26842480128 filled by sysfs probe.
I0827 21:04:23.013232 6 sysfsprobe.go:125] blockdevice path: /dev/sdb1 logical block size :512 filled by sysfs probe.
I0827 21:04:23.013367 6 sysfsprobe.go:137] blockdevice path: /dev/sdb1 physical block size :512 filled by sysfs probe.
I0827 21:04:23.013415 6 sysfsprobe.go:149] blockdevice path: /dev/sdb1 hardware sector size :512 filled by sysfs probe.
I0827 21:04:23.013447 6 sysfsprobe.go:160] blockdevice path: /dev/sdb1 drive type :HDD filled by sysfs probe.
I0827 21:04:23.013453 6 probe.go:119] details filled by sysfs probe
I0827 21:04:23.013981 6 probe.go:119] details filled by smart probe
I0827 21:04:23.014640 6 mountprobe.go:134] no mount point found for /dev/sdb1. clearing mount points if any
I0827 21:04:23.015158 6 probe.go:119] details filled by mount probe
I0827 21:04:23.015417 6 udevprobe.go:403] processing new event for (/dev/sdb1) action type remove
I0827 21:04:23.021136 6 probe.go:119] details filled by blkid probe
I0827 21:04:23.033936 6 probe.go:119] details filled by used-by probe
I0827 21:04:23.033955 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:23.033964 6 addhandler.go:53] device: /dev/sdb1 does not exist in cache, the device is now connected to this node
I0827 21:04:23.033994 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sdb1
I0827 21:04:23.034017 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sdb1
I0827 21:04:23.034041 6 eventhandler.go:82] Processed details for /dev/sdb1
I0827 21:04:23.034049 6 addhandler.go:106] checking if device: /dev/sdb1 can be uniquely identified
I0827 21:04:23.034062 6 uuid.go:73] device(/dev/sdb1) is a partition, using partition UUID: a7687955-8e3f-4357-a50e-95f2e35cb263
I0827 21:04:23.034070 6 uuid.go:101] generated uuid: blockdevice-217d048e4297e01cd76b39589f529a1f for device: /dev/sdb1
I0827 21:04:23.034075 6 addhandler.go:141] uuid: blockdevice-217d048e4297e01cd76b39589f529a1f has been generated for device: /dev/sdb1
I0827 21:04:23.034080 6 addhandler.go:47] device: /dev/sdb1 already exists in cache, the event was likely generated by a partition table re-read or a change in some of the devices was detected
I0827 21:04:23.039504 6 blockdevicestore.go:146] Got blockdevice object : blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:04:23.039527 6 addhandler.go:246] creating resource for device: /dev/sdb1 with uuid: blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:04:23.039567 6 blockdevice.go:155] successfully added device labels
I0827 21:04:23.051124 6 blockdevicestore.go:113] eventcode=ndm.blockdevice.update.success msg=Updated blockdevice object rname=blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:04:23.051287 6 udevprobe.go:403] processing new event for (/dev/sdb1) action type add
I0827 21:04:23.051681 6 udevprobe.go:450] Dependents of /dev/sdb1 : {Parent:/dev/sdb Partitions:[] Holders:[] Slaves:[]}
I0827 21:04:23.051715 6 udevprobe.go:459] Device: /dev/sdb1 is of type: partition
I0827 21:04:23.056452 6 uuid.go:73] device(/dev/sdb1) is a partition, using partition UUID: a7687955-8e3f-4357-a50e-95f2e35cb263
I0827 21:04:23.056481 6 uuid.go:101] generated uuid: blockdevice-217d048e4297e01cd76b39589f529a1f for device: /dev/sdb1
I0827 21:04:23.069294 6 blockdevicestore.go:131] eventcode=ndm.blockdevice.deactivate.success msg=Deactivated blockdevice rname=blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:04:23.069319 6 deletehandler.go:57] deactivated device: /dev/sdb1, using GPT UUID
I0827 21:04:23.073474 6 eventhandler.go:69] Processing details for /dev/sdb1
I0827 21:04:23.073765 6 udevprobe.go:294] device: /dev/sdb1, Model: QEMU_HARDDISK filled by udev probe
I0827 21:04:23.073780 6 udevprobe.go:302] device: /dev/sdb1, Serial: drive-scsi1 filled by udev probe
I0827 21:04:23.073785 6 udevprobe.go:306] device: /dev/sdb1, Vendor: QEMU filled by udev probe
I0827 21:04:23.073791 6 udevprobe.go:310] device: /dev/sdb1, IDType: disk filled by udev probe
I0827 21:04:23.073806 6 probe.go:119] details filled by udev probe
I0827 21:04:23.073915 6 sysfsprobe.go:97] blockdevice path: /dev/sdb1 capacity :26842480128 filled by sysfs probe.
I0827 21:04:23.074008 6 sysfsprobe.go:125] blockdevice path: /dev/sdb1 logical block size :512 filled by sysfs probe.
I0827 21:04:23.074045 6 sysfsprobe.go:137] blockdevice path: /dev/sdb1 physical block size :512 filled by sysfs probe.
I0827 21:04:23.074079 6 sysfsprobe.go:149] blockdevice path: /dev/sdb1 hardware sector size :512 filled by sysfs probe.
I0827 21:04:23.074112 6 sysfsprobe.go:160] blockdevice path: /dev/sdb1 drive type :HDD filled by sysfs probe.
I0827 21:04:23.074118 6 probe.go:119] details filled by sysfs probe
I0827 21:04:23.074785 6 probe.go:119] details filled by smart probe
I0827 21:04:23.075206 6 mountprobe.go:134] no mount point found for /dev/sdb1. clearing mount points if any
I0827 21:04:23.075220 6 probe.go:119] details filled by mount probe
I0827 21:04:23.081090 6 probe.go:119] details filled by blkid probe
I0827 21:04:23.089475 6 probe.go:119] details filled by used-by probe
I0827 21:04:23.089495 6 probe.go:119] details filled by Custom Tag Probe
I0827 21:04:23.089504 6 addhandler.go:53] device: /dev/sdb1 does not exist in cache, the device is now connected to this node
I0827 21:04:23.089551 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sdb1
I0827 21:04:23.089592 6 osdiskexcludefilter.go:129] applying os-filter regex ^/dev/sda[0-9]$ on /dev/sdb1
I0827 21:04:23.089648 6 eventhandler.go:82] Processed details for /dev/sdb1
I0827 21:04:23.089664 6 addhandler.go:106] checking if device: /dev/sdb1 can be uniquely identified
I0827 21:04:23.089671 6 uuid.go:73] device(/dev/sdb1) is a partition, using partition UUID: a7687955-8e3f-4357-a50e-95f2e35cb263
I0827 21:04:23.089679 6 uuid.go:101] generated uuid: blockdevice-217d048e4297e01cd76b39589f529a1f for device: /dev/sdb1
I0827 21:04:23.089684 6 addhandler.go:141] uuid: blockdevice-217d048e4297e01cd76b39589f529a1f has been generated for device: /dev/sdb1
I0827 21:04:23.089689 6 addhandler.go:47] device: /dev/sdb1 already exists in cache, the event was likely generated by a partition table re-read or a change in some of the devices was detected
I0827 21:04:23.093766 6 blockdevicestore.go:146] Got blockdevice object : blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:04:23.093791 6 addhandler.go:246] creating resource for device: /dev/sdb1 with uuid: blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:04:23.093829 6 blockdevice.go:155] successfully added device labels
I0827 21:04:23.103772 6 blockdevicestore.go:113] eventcode=ndm.blockdevice.update.success msg=Updated blockdevice object rname=blockdevice-217d048e4297e01cd76b39589f529a1f
[entrypoint.sh] caught SIGTERM signal forwarding to pid [6].
E0827 21:07:10.922087 6 blockdevicestore.go:278] Status marked unknown for blockdevice object: blockdevice-217d048e4297e01cd76b39589f529a1f
I0827 21:07:10.922109 6 controller.go:330] shutting down the controller
root@k8smaster1:/etc/kubernetes# kubectl logs openebs-ndm-node-exporter-wqsvb -n openebs --previous
E0827 21:04:08.219761 1 logs.go:35] unable to set flag, Error: no such flag -logtostderr
I0827 21:04:08.220009 1 exporter.go:52] Starting NDM Exporter...
I0827 21:04:08.220020 1 exporter.go:53] Version Tag : 2.1.0
I0827 21:04:08.220025 1 exporter.go:54] GitCommit : b9f14b6af7a2f93ab915c43a4ebb402de427000b

oot@k8smaster1:/etc/kubernetes# kubectl get networkpolicies -n openebs -o yaml
apiVersion: v1
items: []
kind: List
metadata:
resourceVersion: ""
@github-actions GitHub Actions
Copy link

Issues go stale after 90d of inactivity. Please comment or re-open the issue if you are still interested in getting this issue fixed.

@avishnu
Copy link
Member

avishnu commented Nov 26, 2023

You may have to check the events, kubectl describe of the pods may help.

@niladrih niladrih added the cstor label Jan 18, 2024
@avishnu
Copy link
Member

avishnu commented Jan 20, 2024

Also, I'd like to suggest Mayastor as the alternative to CStor.
Mayastor is a replicated storage solution (just like CStor) but it's very fast as compared to CStor.
Mayastor is under active development and support, unlike CStor.

https://github.com/openebs/mayastor
https://mayastor.gitbook.io/introduction/

@avishnu avishnu added the Need INFO Need info from the user label Jan 20, 2024
@tidux
Copy link

tidux commented Mar 9, 2024

Mayastor doesn't work on ARM. Abandoning CStor means giving up on ARM. Please update your documentation accordingly if ARM is no longer intended as a support target for OpenEBS beyond local volumes.

@orville-wright orville-wright changed the title Kubernetes - openebs - CrashLoopBackOff and I don't know why JIVA : Kubernetes - openebs - CrashLoopBackOff and I don't know why Apr 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CNCF Community Community Reported Issue Need INFO Need info from the user No Activity - CLOSE issue Will CLOSE in 7 days
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tidux @orville-wright @avishnu @niladrih @RiioDeezy