Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add replaceUnready annotation for Jobs #1575

Merged
merged 4 commits into from
May 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
## HEAD (Unreleased)

- Allow opting out of CRD rendering for Helm v3 by specifying `SkipCRDRendering` argument to Helm charts. (https://github.com/pulumi/pulumi-kubernetes/pull/1572)
- Add replaceUnready annotation for Jobs. (https://github.com/pulumi/pulumi-kubernetes/pull/1575)

## 3.1.2 (May 12, 2021)

Expand Down
4 changes: 2 additions & 2 deletions provider/cmd/pulumi-resource-kubernetes/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -9680,7 +9680,7 @@
}
},
"kubernetes:batch/v1:Job": {
"description": "Job represents the configuration of a single job.\n\nThis resource waits until its status is ready before registering success\nfor create/update, and populating output properties from the current state of the resource.\nThe following conditions are used to determine whether the resource creation has\nsucceeded or failed:\n\n1. The Job's '.status.startTime' is set, which indicates that the Job has started running.\n2. The Job's '.status.conditions' has a status of type 'Complete', and a 'status' set\n to 'True'.\n3. The Job's '.status.conditions' do not have a status of type 'Failed', with a\n\t'status' set to 'True'. If this condition is set, we should fail the Job immediately.\n\nIf the Job has not reached a Ready state after 10 minutes, it will\ntime out and mark the resource update as Failed. You can override the default timeout value\nby setting the 'customTimeouts' option on the resource.",
"description": "Job represents the configuration of a single job.\n\nThis resource waits until its status is ready before registering success\nfor create/update, and populating output properties from the current state of the resource.\nThe following conditions are used to determine whether the resource creation has\nsucceeded or failed:\n\n1. The Job's '.status.startTime' is set, which indicates that the Job has started running.\n2. The Job's '.status.conditions' has a status of type 'Complete', and a 'status' set\n to 'True'.\n3. The Job's '.status.conditions' do not have a status of type 'Failed', with a\n\t'status' set to 'True'. If this condition is set, we should fail the Job immediately.\n\nIf the Job has not reached a Ready state after 10 minutes, it will\ntime out and mark the resource update as Failed. You can override the default timeout value\nby setting the 'customTimeouts' option on the resource.\n\nBy default, if a resource failed to become ready in a previous update, \nPulumi will continue to wait for readiness on the next update. If you would prefer\nto schedule a replacement for an unready resource on the next update, you can add the\n\"pulumi.com/replaceUnready\": \"true\" annotation to the resource definition.",
"properties": {
"apiVersion": {
"type": "string",
Expand Down Expand Up @@ -33131,7 +33131,7 @@
]
},
"kubernetes:batch/v1:Job": {
"description": "Job represents the configuration of a single job.\n\nThis resource waits until its status is ready before registering success\nfor create/update, and populating output properties from the current state of the resource.\nThe following conditions are used to determine whether the resource creation has\nsucceeded or failed:\n\n1. The Job's '.status.startTime' is set, which indicates that the Job has started running.\n2. The Job's '.status.conditions' has a status of type 'Complete', and a 'status' set\n to 'True'.\n3. The Job's '.status.conditions' do not have a status of type 'Failed', with a\n\t'status' set to 'True'. If this condition is set, we should fail the Job immediately.\n\nIf the Job has not reached a Ready state after 10 minutes, it will\ntime out and mark the resource update as Failed. You can override the default timeout value\nby setting the 'customTimeouts' option on the resource.",
"description": "Job represents the configuration of a single job.\n\nThis resource waits until its status is ready before registering success\nfor create/update, and populating output properties from the current state of the resource.\nThe following conditions are used to determine whether the resource creation has\nsucceeded or failed:\n\n1. The Job's '.status.startTime' is set, which indicates that the Job has started running.\n2. The Job's '.status.conditions' has a status of type 'Complete', and a 'status' set\n to 'True'.\n3. The Job's '.status.conditions' do not have a status of type 'Failed', with a\n\t'status' set to 'True'. If this condition is set, we should fail the Job immediately.\n\nIf the Job has not reached a Ready state after 10 minutes, it will\ntime out and mark the resource update as Failed. You can override the default timeout value\nby setting the 'customTimeouts' option on the resource.\n\nBy default, if a resource failed to become ready in a previous update, \nPulumi will continue to wait for readiness on the next update. If you would prefer\nto schedule a replacement for an unready resource on the next update, you can add the\n\"pulumi.com/replaceUnready\": \"true\" annotation to the resource definition.",
"properties": {
"apiVersion": {
"type": "string",
Expand Down
11 changes: 10 additions & 1 deletion provider/pkg/gen/additionalComments.go
Original file line number Diff line number Diff line change
Expand Up @@ -147,15 +147,24 @@ https://kubernetes.io/docs/concepts/configuration/secret/#risks`
}
}

func replaceUnreadyComment() string {
return `By default, if a resource failed to become ready in a previous update,
Pulumi will continue to wait for readiness on the next update. If you would prefer
to schedule a replacement for an unready resource on the next update, you can add the
"pulumi.com/replaceUnready": "true" annotation to the resource definition.`
}

// PulumiComment adds additional information to the docs generated automatically from the OpenAPI specs.
// This includes information about Pulumi's await behavior, deprecation information, etc.
func PulumiComment(kind string) string {
const prefix = "\n\n"

k := kinds.Kind(kind)
switch k {
case kinds.Deployment, kinds.Ingress, kinds.Job, kinds.Pod, kinds.Service, kinds.StatefulSet:
case kinds.Deployment, kinds.Ingress, kinds.Pod, kinds.Service, kinds.StatefulSet:
return prefix + awaitComments(k)
case kinds.Job:
return prefix + awaitComments(k) + prefix + replaceUnreadyComment()
case kinds.Secret:
return prefix + helpfulLinkComments(k)
default:
Expand Down
1 change: 1 addition & 0 deletions provider/pkg/metadata/annotations.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ const (
AnnotationSkipAwait = AnnotationPrefix + "skipAwait"
AnnotationTimeoutSeconds = AnnotationPrefix + "timeoutSeconds"
AnnotationInitialAPIVersion = AnnotationPrefix + "initialApiVersion"
AnnotationReplaceUnready = AnnotationPrefix + "replaceUnready"

AnnotationHelmHook = "helm.sh/hook"
)
Expand Down
5 changes: 5 additions & 0 deletions provider/pkg/metadata/overrides.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ func SkipAwaitLogic(obj *unstructured.Unstructured) bool {
return IsAnnotationTrue(obj, AnnotationSkipAwait)
}

// ReplaceUnready returns true if the `pulumi.com/replaceUnready` annotation is "true", false otherwise.
func ReplaceUnready(obj *unstructured.Unstructured) bool {
return IsAnnotationTrue(obj, AnnotationReplaceUnready)
}

// TimeoutDuration returns the resource timeout duration. There are a number of things it can do here in this order
// 1. Return the timeout as specified in the customResource options
// 2. Return the timeout as specified in `pulumi.com/timeoutSeconds` annotation,
Expand Down
20 changes: 19 additions & 1 deletion provider/pkg/provider/provider.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright 2016-2018, Pulumi Corporation.
// Copyright 2016-2021, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -35,6 +35,7 @@ import (
structpb "github.com/golang/protobuf/ptypes/struct"
pkgerrors "github.com/pkg/errors"
"github.com/pulumi/pulumi-kubernetes/provider/v3/pkg/await"
"github.com/pulumi/pulumi-kubernetes/provider/v3/pkg/await/states"
"github.com/pulumi/pulumi-kubernetes/provider/v3/pkg/clients"
"github.com/pulumi/pulumi-kubernetes/provider/v3/pkg/cluster"
"github.com/pulumi/pulumi-kubernetes/provider/v3/pkg/gen"
Expand Down Expand Up @@ -1368,6 +1369,23 @@ func (k *kubeProvider) Diff(
}
}

if metadata.ReplaceUnready(newInputs) {
switch newInputs.GetKind() {
case "Job":
jobChecker := states.NewJobChecker()
job, err := clients.FromUnstructured(newInputs)
if err == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it intentional that the error is swallowed here? Shouldn't it be returned?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this error would only occur from a malformed Job object and is unrelated to the diff, I thought it made sense to ignore. I can add a comment to that effect.

jobChecker.Update(job)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it okay to do this wait operation from Diff? Do we normally do that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not actually running the full await logic; this is a point in time check based on the current status of the Job.

I'm not sure if there's precedent for that, but it shouldn't have a noticeable effect on performance, and Diff is the only place where we can indicate a replacement is required.

if !jobChecker.Ready() {
hasChanges = pulumirpc.DiffResponse_DIFF_SOME
replaces = append(replaces, `.metadata.annotations["pulumi.com/replaceUnready"]`)
}
}
default:
_ = k.host.Log(ctx, diag.Warning, urn, "replaceUnready annotation is not supported for this resource")
}
lblackstone marked this conversation as resolved.
Show resolved Hide resolved
}

// Delete before replacement if we are forced to replace the old object, and the new version of
// that object MUST have the same name.
deleteBeforeReplace :=
Expand Down
5 changes: 5 additions & 0 deletions sdk/dotnet/Batch/V1/Job.cs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ namespace Pulumi.Kubernetes.Batch.V1
/// If the Job has not reached a Ready state after 10 minutes, it will
/// time out and mark the resource update as Failed. You can override the default timeout value
/// by setting the 'customTimeouts' option on the resource.
///
/// By default, if a resource failed to become ready in a previous update,
/// Pulumi will continue to wait for readiness on the next update. If you would prefer
/// to schedule a replacement for an unready resource on the next update, you can add the
/// "pulumi.com/replaceUnready": "true" annotation to the resource definition.
/// </summary>
[KubernetesResourceType("kubernetes:batch/v1:Job")]
public partial class Job : KubernetesResource
Expand Down
5 changes: 5 additions & 0 deletions sdk/go/kubernetes/batch/v1/job.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ import (
// If the Job has not reached a Ready state after 10 minutes, it will
// time out and mark the resource update as Failed. You can override the default timeout value
// by setting the 'customTimeouts' option on the resource.
//
// By default, if a resource failed to become ready in a previous update,
// Pulumi will continue to wait for readiness on the next update. If you would prefer
// to schedule a replacement for an unready resource on the next update, you can add the
// "pulumi.com/replaceUnready": "true" annotation to the resource definition.
type Job struct {
pulumi.CustomResourceState

Expand Down
15 changes: 15 additions & 0 deletions sdk/go/kubernetes/batch/v1/pulumiTypes.go
Original file line number Diff line number Diff line change
Expand Up @@ -666,6 +666,11 @@ func (o CronJobStatusPtrOutput) LastSuccessfulTime() pulumi.StringPtrOutput {
// If the Job has not reached a Ready state after 10 minutes, it will
// time out and mark the resource update as Failed. You can override the default timeout value
// by setting the 'customTimeouts' option on the resource.
//
// By default, if a resource failed to become ready in a previous update,
// Pulumi will continue to wait for readiness on the next update. If you would prefer
// to schedule a replacement for an unready resource on the next update, you can add the
// "pulumi.com/replaceUnready": "true" annotation to the resource definition.
type JobType struct {
// APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
ApiVersion *string `pulumi:"apiVersion"`
Expand Down Expand Up @@ -706,6 +711,11 @@ type JobTypeInput interface {
// If the Job has not reached a Ready state after 10 minutes, it will
// time out and mark the resource update as Failed. You can override the default timeout value
// by setting the 'customTimeouts' option on the resource.
//
// By default, if a resource failed to become ready in a previous update,
// Pulumi will continue to wait for readiness on the next update. If you would prefer
// to schedule a replacement for an unready resource on the next update, you can add the
// "pulumi.com/replaceUnready": "true" annotation to the resource definition.
type JobTypeArgs struct {
// APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
ApiVersion pulumi.StringPtrInput `pulumi:"apiVersion"`
Expand Down Expand Up @@ -772,6 +782,11 @@ func (i JobTypeArray) ToJobTypeArrayOutputWithContext(ctx context.Context) JobTy
// If the Job has not reached a Ready state after 10 minutes, it will
// time out and mark the resource update as Failed. You can override the default timeout value
// by setting the 'customTimeouts' option on the resource.
//
// By default, if a resource failed to become ready in a previous update,
// Pulumi will continue to wait for readiness on the next update. If you would prefer
// to schedule a replacement for an unready resource on the next update, you can add the
// "pulumi.com/replaceUnready": "true" annotation to the resource definition.
type JobTypeOutput struct{ *pulumi.OutputState }

func (JobTypeOutput) ElementType() reflect.Type {
Expand Down
5 changes: 5 additions & 0 deletions sdk/nodejs/batch/v1/job.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ import * as utilities from "../../utilities";
* If the Job has not reached a Ready state after 10 minutes, it will
* time out and mark the resource update as Failed. You can override the default timeout value
* by setting the 'customTimeouts' option on the resource.
*
* By default, if a resource failed to become ready in a previous update,
* Pulumi will continue to wait for readiness on the next update. If you would prefer
* to schedule a replacement for an unready resource on the next update, you can add the
* "pulumi.com/replaceUnready": "true" annotation to the resource definition.
*/
export class Job extends pulumi.CustomResource {
/**
Expand Down
5 changes: 5 additions & 0 deletions sdk/nodejs/types/input.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4831,6 +4831,11 @@ export namespace batch {
* If the Job has not reached a Ready state after 10 minutes, it will
* time out and mark the resource update as Failed. You can override the default timeout value
* by setting the 'customTimeouts' option on the resource.
*
* By default, if a resource failed to become ready in a previous update,
* Pulumi will continue to wait for readiness on the next update. If you would prefer
* to schedule a replacement for an unready resource on the next update, you can add the
* "pulumi.com/replaceUnready": "true" annotation to the resource definition.
*/
export interface Job {
/**
Expand Down
5 changes: 5 additions & 0 deletions sdk/nodejs/types/output.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5091,6 +5091,11 @@ export namespace batch {
* If the Job has not reached a Ready state after 10 minutes, it will
* time out and mark the resource update as Failed. You can override the default timeout value
* by setting the 'customTimeouts' option on the resource.
*
* By default, if a resource failed to become ready in a previous update,
* Pulumi will continue to wait for readiness on the next update. If you would prefer
* to schedule a replacement for an unready resource on the next update, you can add the
* "pulumi.com/replaceUnready": "true" annotation to the resource definition.
*/
export interface Job {
/**
Expand Down
10 changes: 10 additions & 0 deletions sdk/python/pulumi_kubernetes/batch/v1/Job.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,11 @@ def __init__(__self__,
time out and mark the resource update as Failed. You can override the default timeout value
by setting the 'customTimeouts' option on the resource.

By default, if a resource failed to become ready in a previous update,
Pulumi will continue to wait for readiness on the next update. If you would prefer
to schedule a replacement for an unready resource on the next update, you can add the
"pulumi.com/replaceUnready": "true" annotation to the resource definition.

:param str resource_name: The name of the resource.
:param pulumi.ResourceOptions opts: Options for the resource.
:param pulumi.Input[str] api_version: APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
Expand Down Expand Up @@ -145,6 +150,11 @@ def __init__(__self__,
time out and mark the resource update as Failed. You can override the default timeout value
by setting the 'customTimeouts' option on the resource.

By default, if a resource failed to become ready in a previous update,
Pulumi will continue to wait for readiness on the next update. If you would prefer
to schedule a replacement for an unready resource on the next update, you can add the
"pulumi.com/replaceUnready": "true" annotation to the resource definition.

:param str resource_name: The name of the resource.
:param JobArgs args: The arguments to use to populate this resource's properties.
:param pulumi.ResourceOptions opts: Options for the resource.
Expand Down
5 changes: 5 additions & 0 deletions sdk/python/pulumi_kubernetes/batch/v1/_inputs.py
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,11 @@ def __init__(__self__, *,
If the Job has not reached a Ready state after 10 minutes, it will
time out and mark the resource update as Failed. You can override the default timeout value
by setting the 'customTimeouts' option on the resource.

By default, if a resource failed to become ready in a previous update,
Pulumi will continue to wait for readiness on the next update. If you would prefer
to schedule a replacement for an unready resource on the next update, you can add the
"pulumi.com/replaceUnready": "true" annotation to the resource definition.
:param pulumi.Input[str] api_version: APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
:param pulumi.Input[str] kind: Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
:param pulumi.Input['_meta.v1.ObjectMetaArgs'] metadata: Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
Expand Down
Loading