Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

invalid memory address or nil pointer dereference" #30373

Closed
sadiapoddar opened this issue Dec 8, 2020 · 4 comments
Closed

invalid memory address or nil pointer dereference" #30373

sadiapoddar opened this issue Dec 8, 2020 · 4 comments
Assignees
@superseb @sadiapoddar
Labels
kind/bug-qa Issues that have not yet hit a real release. Bugs introduced by a new feature or enhancement status/blocker
Milestone
v2.5.4

Comments

@sadiapoddar
Copy link

sadiapoddar commented Dec 8, 2020 •
edited

What kind of request is this (question/bug/enhancement/feature request):
Bug

Steps to reproduce (least amount of steps as possible):
2.5-head, commit 6ad4dce

  1. Initiated 4 nodes(3woker,1etc+control) node driver cluster
  2. While the cluster provision was continuing
  3. From UI started to create a custom cluster (rke) and
  4. UI generated the command to register the existing nodes as custom cluster
  5. Executed the UI generated command on an ec2 instance to register as "etcd + control plane" node
  6. Noticed that Rancher UI is not reachable anymore
  7. Found the null pointer error
    ----------- docker logs -----------------
    k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00160d700)
    /go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:155 +0x5f
    k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00160d700, 0x4e296e0, 0xc00266bb90, 0x727301, 0xc0014822a0)
    /go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:156 +0xa3
    k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00160d700, 0x3b9aca00, 0x0, 0x46c4201, 0xc0014822a0)
    /go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:133 +0x98
    k8s.io/apimachinery/pkg/util/wait.Until(0xc00160d700, 0x3b9aca00, 0xc0014822a0)
    /go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:90 +0x4d
    created by github.com/rancher/lasso/pkg/controller.(*controller).run
    /go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:129 +0x31f
    panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x1 pc=0x4c30f4]

goroutine 1159 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/runtime/runtime.go:55 +0x105
panic(0x3c3bce0, 0x73b0e40)
/usr/local/go/src/runtime/panic.go:969 +0x166
encoding/json.(*encodeState).marshal.func1(0xc0024c8d50)
/usr/local/go/src/encoding/json/encode.go:326 +0x9a
panic(0x3c3bce0, 0x73b0e40)
/usr/local/go/src/runtime/panic.go:969 +0x166
reflect.Value.String(0x392fe60, 0x1, 0x198, 0x4f3e400, 0x392fe60)
/usr/local/go/src/reflect/value.go:1847 +0x44
encoding/json.stringEncoder(0xc002774d00, 0x392fe60, 0x1, 0x198, 0x100)
/usr/local/go/src/encoding/json/encode.go:645 +0x18d
encoding/json.structEncoder.encode(0xc000ad2900, 0x4, 0x4, 0xc0067f7e00, 0xc002774d00, 0x413e3e0, 0x1, 0x199, 0x100)
/usr/local/go/src/encoding/json/encode.go:759 +0x2bb
encoding/json.arrayEncoder.encode(0xc0067f7e30, 0xc002774d00, 0x38639c0, 0xc006c2dc60, 0x97, 0x760100)
/usr/local/go/src/encoding/json/encode.go:886 +0xd6
encoding/json.sliceEncoder.encode(0xc006c18c10, 0xc002774d00, 0x38639c0, 0xc006c2dc60, 0x97, 0x3860100)
/usr/local/go/src/encoding/json/encode.go:860 +0x8f
encoding/json.(*encodeState).reflectValue(0xc002774d00, 0x38639c0, 0xc006c2dc60, 0x97, 0xc0024c0100)
/usr/local/go/src/encoding/json/encode.go:358 +0x82
encoding/json.(*encodeState).marshal(0xc002774d00, 0x38639c0, 0xc006c2dc60, 0x100, 0x0, 0x0)
/usr/local/go/src/encoding/json/encode.go:330 +0xf0
encoding/json.Marshal(0x38639c0, 0xc006c2dc60, 0x3a63ae0, 0x35cb9d797f2f3554, 0x9d5ef009c45, 0x8, 0x18)
/usr/local/go/src/encoding/json/encode.go:161 +0x52
github.com/rancher/rke/templates.ToYAML(0x38639c0, 0xc006c2dc60, 0x15, 0xc006c2dc60)
/go/pkg/mod/github.com/rancher/rke@v1.2.0-rc9.0.20201204145714-816d4cd130a9/templates/templates.go:44 +0x4d
github.com/rancher/rancher/pkg/systemtemplate.SystemTemplate(0x4e236a0, 0xc006b99500, 0xc000074013, 0x48, 0xc00138a5e0, 0x1c, 0xc00257c650, 0x7, 0xc007bddfc0, 0x36, ...)
/go/src/github.com/rancher/rancher/pkg/systemtemplate/import.go:85 +0x49b
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).getYAML(0xc007bce8c0, 0xc0035d2000, 0xc000074013, 0x48, 0xc00138a5e0, 0x1c, 0xc006bb08a0, 0x41ddc20, 0xc006814c20, 0x199, ...)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:371 +0x455
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).deployAgent.func1(0x4458460, 0xc0035d2000, 0x4504e95, 0xd)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:255 +0x89
github.com/rancher/norman/condition.Cond.doInternal(0x4504e95, 0xd, 0x4e53c01, 0x4e53ca0, 0xc0035d2000, 0xc0024c9800, 0xc006bb0990, 0xc00785b220, 0xc006bb0960, 0xc003a77f00)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:212 +0x97
github.com/rancher/norman/condition.Cond.do2(0x4504e95, 0xd, 0xc008a4fb01, 0x4e53ca0, 0xc0035d2000, 0xc0024c9800, 0x1da8c10, 0xc0062e2340, 0xc00306dee0, 0x392fe60, ...)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:181 +0x1d8
github.com/rancher/norman/condition.Cond.do(...)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:170
github.com/rancher/norman/condition.Cond.Do(0x4504e95, 0xd, 0x4e53ca0, 0xc0035d2000, 0xc0024c9800, 0xc006bb08a0, 0x1, 0xc0024c9710, 0x12c7d87)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:140 +0x62
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).deployAgent(0xc007bce8c0, 0xc0035d2000, 0x4e53ca0, 0xc0035d2000)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:254 +0x617
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).doSync(0xc007bce8c0, 0xc0035d2000, 0x461a144, 0x74)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:138 +0x3ba
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).sync(0xc007bce8c0, 0xc00257c650, 0x7, 0xc00112f500, 0xc000100000, 0x7f49456807d0, 0x0, 0x0)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:94 +0x1b9
github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3.(*clusterController).AddHandler.func1(0xc00257c650, 0x7, 0x4458460, 0xc00112f500, 0xc0024c9b20, 0x3, 0x3, 0x0)
/go/src/github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/zz_generated_cluster_controller.go:155 +0x63
github.com/rancher/norman/controller.(*genericController).AddHandler.func1(0xc00257c650, 0x7, 0x4e53ca0, 0xc00112f500, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/controller/generic_controller.go:62 +0x221
github.com/rancher/lasso/pkg/controller.SharedControllerHandlerFunc.OnChange(0xc007bcd860, 0xc00257c650, 0x7, 0x4e53ca0, 0xc00112f500, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/sharedcontroller.go:29 +0x4e
github.com/rancher/lasso/pkg/controller.(*sharedHandler).OnChange(0xc0017b0180, 0xc00257c650, 0x7, 0x4e53ca0, 0xc00112f500, 0xc002890a01, 0x0)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/sharedhandler.go:65 +0x14a
github.com/rancher/lasso/pkg/controller.(*controller).syncHandler(0xc000b00e70, 0xc00257c650, 0x7, 0xc00266b920, 0xc004ed3d50)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:210 +0xd1
github.com/rancher/lasso/pkg/controller.(*controller).processSingleItem(0xc000b00e70, 0x392fe60, 0xc0075bab40, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:192 +0xd7
github.com/rancher/lasso/pkg/controller.(*controller).processNextWorkItem(0xc000b00e70, 0x203001)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:169 +0x51
github.com/rancher/lasso/pkg/controller.(*controller).runWorker(0xc000b00e70)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:158 +0x2b
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00160d700)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00160d700, 0x4e296e0, 0xc00266bb90, 0x727301, 0xc0014822a0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:156 +0xa3
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00160d700, 0x3b9aca00, 0x0, 0x46c4201, 0xc0014822a0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(0xc00160d700, 0x3b9aca00, 0xc0014822a0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:90 +0x4d
created by github.com/rancher/lasso/pkg/controller.(*controller).run
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:129 +0x31f

Result:
Rancher UI is not available

Environment information

  • Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):2.5-head
  • Installation option (single install/HA): single install

Cluster information

  • Cluster type (Hosted/Infrastructure Provider/Custom/Imported): custom
  • Machine type (cloud/VM/metal) and specifications (CPU/memory): N/A
  • Kubernetes version (use kubectl version): v1.19
@sowmyav27 sowmyav27 added this to the v2.5.4 milestone Dec 8, 2020
@sowmyav27 sowmyav27 added the kind/bug-qa Issues that have not yet hit a real release. Bugs introduced by a new feature or enhancement label Dec 8, 2020
@sadiapoddar
Copy link
Author

I can consistently repro this issue by just creating a node driver cluster, 2.5-head, commit 6ad4dce
Steps to repro

  1. Select "add cluster"
  2. select cluster type "With RKE and new nodes in an infrastructure provider"
  3. select "amazon ec2"
  4. select to create 3 workers and 1 (etcd+control) node
  5. click on create to provision the cluster,
  6. after a few sec later we will see UI is not reachable and
  7. Docker log shows the go client panic

020/12/08 03:05:20 [INFO] [mgmt-auth-crtb-controller] Creating clusterRoleBinding for membership in cluster c-jrg7w for subject u-qazomwr4m5
E1208 03:05:20.165881 7 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 1221 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x3c3bce0, 0x73b0e40)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/runtime/runtime.go:74 +0xa3
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/runtime/runtime.go:48 +0x82
panic(0x3c3bce0, 0x73b0e40)
/usr/local/go/src/runtime/panic.go:969 +0x166
encoding/json.(*encodeState).marshal.func1(0xc008066d50)
/usr/local/go/src/encoding/json/encode.go:326 +0x9a
panic(0x3c3bce0, 0x73b0e40)
/usr/local/go/src/runtime/panic.go:969 +0x166
reflect.Value.String(0x392fe60, 0x1, 0x198, 0x4f3e400, 0x392fe60)
/usr/local/go/src/reflect/value.go:1847 +0x44
encoding/json.stringEncoder(0xc00561e880, 0x392fe60, 0x1, 0x198, 0x100)
/usr/local/go/src/encoding/json/encode.go:645 +0x18d
encoding/json.structEncoder.encode(0xc000a91200, 0x4, 0x4, 0xc0038f0060, 0xc00561e880, 0x413e3e0, 0x1, 0x199, 0x100)
/usr/local/go/src/encoding/json/encode.go:759 +0x2bb
encoding/json.arrayEncoder.encode(0xc0038f0090, 0xc00561e880, 0x38639c0, 0xc00a140300, 0x97, 0x760100)
/usr/local/go/src/encoding/json/encode.go:886 +0xd6
encoding/json.sliceEncoder.encode(0xc003170c00, 0xc00561e880, 0x38639c0, 0xc00a140300, 0x97, 0x3860100)
/usr/local/go/src/encoding/json/encode.go:860 +0x8f
encoding/json.(*encodeState).reflectValue(0xc00561e880, 0x38639c0, 0xc00a140300, 0x97, 0xc008060100)
/usr/local/go/src/encoding/json/encode.go:358 +0x82
encoding/json.(*encodeState).marshal(0xc00561e880, 0x38639c0, 0xc00a140300, 0x100, 0x0, 0x0)
/usr/local/go/src/encoding/json/encode.go:330 +0xf0
encoding/json.Marshal(0x38639c0, 0xc00a140300, 0x3a63ae0, 0xed8234d0f6118c1e, 0x3b16249d2b6c, 0x8, 0x18)
/usr/local/go/src/encoding/json/encode.go:161 +0x52
github.com/rancher/rke/templates.ToYAML(0x38639c0, 0xc00a140300, 0x15, 0xc00a140300)
/go/pkg/mod/github.com/rancher/rke@v1.2.0-rc9.0.20201204145714-816d4cd130a9/templates/templates.go:44 +0x4d
github.com/rancher/rancher/pkg/systemtemplate.SystemTemplate(0x4e236a0, 0xc00aa5e090, 0xc000074013, 0x48, 0xc000ed3a60, 0x1c, 0xc00894aa04, 0x7, 0xc006cb9080, 0x36, ...)
/go/src/github.com/rancher/rancher/pkg/systemtemplate/import.go:85 +0x49b
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).getYAML(0xc0054670c0, 0xc005598700, 0xc000074013, 0x48, 0xc000ed3a60, 0x1c, 0xc00bd5bd70, 0x41ddc20, 0xc001e97420, 0x199, ...)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:371 +0x455
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).deployAgent.func1(0x4458460, 0xc005598700, 0x4504e95, 0xd)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:255 +0x89
github.com/rancher/norman/condition.Cond.doInternal(0x4504e95, 0xd, 0x4e53c01, 0x4e53ca0, 0xc005598700, 0xc008067800, 0xc008bd1500, 0xc007176e10, 0xc008bd14d0, 0xc004da4a00)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:212 +0x97
github.com/rancher/norman/condition.Cond.do2(0x4504e95, 0xd, 0xc001db4601, 0x4e53ca0, 0xc005598700, 0xc008067800, 0x1da8c10, 0xc00505b520, 0xc001835860, 0x392fe60, ...)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:181 +0x1d8
github.com/rancher/norman/condition.Cond.do(...)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:170
github.com/rancher/norman/condition.Cond.Do(0x4504e95, 0xd, 0x4e53ca0, 0xc005598700, 0xc008067800, 0xc00bd5bd70, 0x1, 0xc008067710, 0x12c7d87)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:140 +0x62
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).deployAgent(0xc0054670c0, 0xc005598700, 0x4e53ca0, 0xc005598700)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:254 +0x617
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).doSync(0xc0054670c0, 0xc005598700, 0x461a144, 0x74)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:138 +0x3ba
github.com/rancher/rancher/pkg/controllers/management/clusterdeploy.(*clusterDeploy).sync(0xc0054670c0, 0xc004667eb0, 0x7, 0xc005598000, 0xc000756e00, 0x7f845c91de98, 0x0, 0x0)
/go/src/github.com/rancher/rancher/pkg/controllers/management/clusterdeploy/clusterdeploy.go:94 +0x1b9
github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3.(*clusterController).AddHandler.func1(0xc004667eb0, 0x7, 0x4458460, 0xc005598000, 0xc008067b20, 0x3, 0x3, 0x0)
/go/src/github.com/rancher/rancher/pkg/generated/norman/management.cattle.io/v3/zz_generated_cluster_controller.go:155 +0x63
github.com/rancher/norman/controller.(*genericController).AddHandler.func1(0xc004667eb0, 0x7, 0x4e53ca0, 0xc005598000, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/controller/generic_controller.go:62 +0x221
github.com/rancher/lasso/pkg/controller.SharedControllerHandlerFunc.OnChange(0xc004f2ddd0, 0xc004667eb0, 0x7, 0x4e53ca0, 0xc005598000, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/sharedcontroller.go:29 +0x4e
github.com/rancher/lasso/pkg/controller.(*sharedHandler).OnChange(0xc000e928a0, 0xc004667eb0, 0x7, 0x4e53ca0, 0xc005598000, 0xc001872401, 0x0)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/sharedhandler.go:65 +0x14a
github.com/rancher/lasso/pkg/controller.(*controller).syncHandler(0xc000815b80, 0xc004667eb0, 0x7, 0xc001c63fb0, 0xc008067d50)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:210 +0xd1
github.com/rancher/lasso/pkg/controller.(*controller).processSingleItem(0xc000815b80, 0x392fe60, 0xc002cf1150, 0x0, 0x0)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:192 +0xd7
github.com/rancher/lasso/pkg/controller.(*controller).processNextWorkItem(0xc000815b80, 0x203001)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:169 +0x51
github.com/rancher/lasso/pkg/controller.(*controller).runWorker(0xc000815b80)
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:158 +0x2b
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00027a280)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:155 +0x5f
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00027a280, 0x4e296e0, 0xc002036870, 0x46c2201, 0xc000dc24e0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:156 +0xa3
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00027a280, 0x3b9aca00, 0x0, 0x467f01, 0xc000dc24e0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:133 +0x98
k8s.io/apimachinery/pkg/util/wait.Until(0xc00027a280, 0x3b9aca00, 0xc000dc24e0)
/go/pkg/mod/github.com/rancher/apimachinery@v0.19.0-rancher1/pkg/util/wait/wait.go:90 +0x4d
created by github.com/rancher/lasso/pkg/controller.(*controller).run
/go/pkg/mod/github.com/rancher/lasso@v0.0.0-20200905045615-7fcb07d6a20b/pkg/controller/controller.go:129 +0x31f
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x1 pc=0x4c30f4]

@superseb superseb assigned superseb and unassigned dramich Dec 8, 2020
@superseb superseb mentioned this issue Dec 8, 2020
Merged
@sowmyav27
Copy link
Contributor

Unable to bring up a ha setup using rancher 2.5-head - commit id: 6ad4dce7

Rancher keeps restarting

@superseb superseb mentioned this issue Dec 8, 2020
Merged
@sadiapoddar
Copy link
Author

Verified the fix 2.5-head, commit b2953cb and
master-head commit ff16903
Steps used to verify

  1. Select "add cluster"
  2. select cluster type "With RKE and new nodes in an infrastructure provider"
  3. select "amazon ec2"
  4. select to create 3 workers and 1 (etcd+control) node
  5. click on create to provision the cluster,
  6. after a few sec later we will see UI is not reachable and

Cluster provisioned without error and cluster is up and running.
Verified Docker log and there was no go client panic

@sowmyav27
Copy link
Contributor

On 2.5-head HA setup - commit id: b2953cb. No restarts seen. A downstream cluster is deployed successfully.

@zube zube bot removed the [zube]: Done label Mar 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@superseb superseb

@sadiapoddar sadiapoddar

Labels
kind/bug-qa Issues that have not yet hit a real release. Bugs introduced by a new feature or enhancement status/blocker
Projects
None yet
Milestone
v2.5.4
Development

No branches or pull requests
5 participants
@maggieliu @superseb @dramich @sowmyav27 @sadiapoddar