MULTIARCH-3669: Avoid setting VM provider ID as soon as VM creation

[Karthik-K-N]

Currently we are trying to set provider id for vm as soon as vm creation call and which is resulting in error and requeing the request.

On an average it takes around 17-18seconds for the created vm to be seen in power clients GetAllInstances(),

Since we are requeing withing this time the controller will not see the previous instance creation call and tries to create one more VM, By this time the previous attempt to create VM is seen.

So the second attempt will fail with error saying "Vm with name already exist"

Error on setting provider id

I0605 11:00:43.102741       1 reconciler.go:77] Created Machine rdr-praj-ipi-414-lon-6266k-worker-b9v45
E0605 11:00:43.102790       1 reconciler.go:239] serviceInstanceID is empty, Cannot set providerID
I0605 11:00:43.102807       1 machine_scope.go:112] rdr-praj-ipi-414-lon-6266k-worker-b9v45: patching machine
E0605 11:00:43.136868       1 actuator.go:77] rdr-praj-ipi-414-lon-6266k-worker-b9v45 error: rdr-praj-ipi-414-lon-6266k-worker-b9v45: reconciler failed to Create machine: failed to update machine object with providerID: serviceInstanceID is empty, Cannot set providerID
W0605 11:00:43.137040       1 controller.go:351] rdr-praj-ipi-414-lon-6266k-worker-b9v45: failed to create machine: rdr-praj-ipi-414-lon-6266k-worker-b9v45: reconciler failed to Create machine: failed to update machine object with providerID: serviceInstanceID is empty, Cannot set providerID
E0605 11:00:43.137282       1 controller.go:329]  "msg"="Reconciler error" "error"="rdr-praj-ipi-414-lon-6266k-worker-b9v45: reconciler failed to Create machine: failed to update machine object with providerID: serviceInstanceID is empty, Cannot set providerID" "controller"="machine-controller" "name"="rdr-praj-ipi-414-lon-6266k-worker-b9v45" "namespace"="openshift-machine-api" "object"={"name":"rdr-praj-ipi-414-lon-6266k-worker-b9v45","namespace":"openshift-machine-api"} "reconcileID"="653e8467-7ef9-4815-b9a3-afcb02806d7e"

Fixes: https://github.ibm.com/redstack-power/project-mgmt/issues/3466

[openshift-ci]

@Karthik-K-N: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[JoelSpeed]

We've done something similar on other platforms, where we only set the provider ID once we have seen that the cloud returns that the VM exists, so if this works for you then it's not out of the ordinary comparing to other providers.

[mkumatag]

This is happening because we are reading the field serviceInstanceID from the ProviderStatus but that is not yet patched into status at that point of time(but present in the machine scope struct). I'm fine with this change to avoid reading this field itself.

[mkumatag]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Karthik-K-N, mkumatag

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Karthik-K-N,mkumatag]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mkumatag]

@Karthik-K-N @pravin-dsilva @juliemathew lets create a valid jira ticket for this and assign..

[openshift-ci-robot]

@Karthik-K-N: This pull request references MULTIARCH-3669 which is a valid jira issue.

In response to this:

Currently we are trying to set provider id for vm as soon as vm creation call and which is resulting in error and requeing the request.

On an average it takes around 17-18seconds for the created vm to be seen in power clients GetAllInstances(),

Since we are requeing withing this time the controller will not see the previous instance creation call and tries to create one more VM, By this time the previous attempt to create VM is seen.

So the second attempt will fail with error saying "Vm with name already exist"

Error on setting provider id

I0605 11:00:43.102741       1 reconciler.go:77] Created Machine rdr-praj-ipi-414-lon-6266k-worker-b9v45
E0605 11:00:43.102790       1 reconciler.go:239] serviceInstanceID is empty, Cannot set providerID
I0605 11:00:43.102807       1 machine_scope.go:112] rdr-praj-ipi-414-lon-6266k-worker-b9v45: patching machine
E0605 11:00:43.136868       1 actuator.go:77] rdr-praj-ipi-414-lon-6266k-worker-b9v45 error: rdr-praj-ipi-414-lon-6266k-worker-b9v45: reconciler failed to Create machine: failed to update machine object with providerID: serviceInstanceID is empty, Cannot set providerID
W0605 11:00:43.137040       1 controller.go:351] rdr-praj-ipi-414-lon-6266k-worker-b9v45: failed to create machine: rdr-praj-ipi-414-lon-6266k-worker-b9v45: reconciler failed to Create machine: failed to update machine object with providerID: serviceInstanceID is empty, Cannot set providerID
E0605 11:00:43.137282       1 controller.go:329]  "msg"="Reconciler error" "error"="rdr-praj-ipi-414-lon-6266k-worker-b9v45: reconciler failed to Create machine: failed to update machine object with providerID: serviceInstanceID is empty, Cannot set providerID" "controller"="machine-controller" "name"="rdr-praj-ipi-414-lon-6266k-worker-b9v45" "namespace"="openshift-machine-api" "object"={"name":"rdr-praj-ipi-414-lon-6266k-worker-b9v45","namespace":"openshift-machine-api"} "reconcileID"="653e8467-7ef9-4815-b9a3-afcb02806d7e"

Fixes: https://github.ibm.com/redstack-power/project-mgmt/issues/3466

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[deepsm007]

/label jira/valid-bug