Skip to content

Upstream: <880>: openshift: Record machine sync error event #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 10, 2019
Merged

Upstream: <880>: openshift: Record machine sync error event #24

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion pkg/controller/machineset/controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,10 @@ func (r *ReconcileMachineSet) reconcile(ctx context.Context, machineSet *machine
return reconcile.Result{}, errors.Wrap(err, "failed to update machine set status")
}

if syncErr != nil {
Copy link

@bison bison Apr 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we only handle this after the updateMachineSetStatus bit? Are those always tied together this way? If so, should that just be a part of syncReplicas?

Copy link
Author

@vikaschoudhary16 vikaschoudhary16 Apr 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MachinesetStatus and events are two mechanisms through which user can get to know about current state. Even if syncReplicas is facing error, i thought it would help if we let the stats updated in the machinesetstatus.

Copy link

@frobware frobware Apr 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a purely pattern perspective, why do we not check for a syncErr straight after we call:

syncErr := r.syncReplicas(machineSet, filteredMachines)

Copy link

@frobware frobware Apr 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If calling syncReplicas can fail why do we not fail fast when this is true?

Copy link
Author

@vikaschoudhary16 vikaschoudhary16 Apr 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kubernetes-sigs#880 (comment)

Copy link

@bison bison Apr 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed this offline. I think it might make sense to just move the updateMachineSetStatus call to syncReplicas if we always want to update the status while syncing. Not a big deal though, and this merged as-is upstream, so LGTM.

Copy link

@frobware frobware Apr 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kubernetes-sigs#880 (comment)

In which case I would say add those exact words as to why we delay checking for success/failure on this call.

Copy link
Author

@vikaschoudhary16 vikaschoudhary16 Apr 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so that machineset status status could get updated and along with error message in the events, user could have latest stats.

return reconcile.Result{}, errors.Wrapf(syncErr, "failed to sync machines")
}

var replicas int32
if updatedMS.Spec.Replicas != nil {
replicas = *updatedMS.Spec.Replicas
Expand All @@ -277,7 +281,7 @@ func (r *ReconcileMachineSet) reconcile(ctx context.Context, machineSet *machine
// exceeds MinReadySeconds could be incorrect.
// To avoid an available replica stuck in the ready state, we force a reconcile after MinReadySeconds,
// at which point it should confirm any available replica to be available.
if syncErr == nil && updatedMS.Spec.MinReadySeconds > 0 &&
if updatedMS.Spec.MinReadySeconds > 0 &&
updatedMS.Status.ReadyReplicas == replicas &&
updatedMS.Status.AvailableReplicas != replicas {

Expand Down