Skip to content

NO-JIRA: Extend HPA test timeout #2528

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
openshift-merge-bot merged 1 commit into from
Dec 8, 2025
Merged

NO-JIRA: Extend HPA test timeout #2528

openshift-merge-bot merged 1 commit into from
Dec 8, 2025

Conversation

@maxcao13
Copy link
Member

@maxcao13 maxcao13 commented Dec 4, 2025

HPA tests are flaking because some are slow enough that they reach the 15 minute default timeout.
This PR sets the default test timeout of the HPA suite to 30 minutes to accommodate the slow tests.

HPA:

	// These tests take ~20 minutes each.
	f.Describe("ReplicationController", func() {
		ginkgo.It(titleUp+" and verify decision stability", func(ctx context.Context) {
			scaleUp(ctx, "rc", e2eautoscaling.KindRC, cpuResource, utilizationMetricType, true, f)
		})
		ginkgo.It(titleDown+" and verify decision stability", func(ctx context.Context) {
			scaleDown(ctx, "rc", e2eautoscaling.KindRC, cpuResource, utilizationMetricType, true, f)
		})
	})

openshift/origin:

	timeout := o.Timeout
	if timeout == 0 {
		timeout = suite.TestTimeout
	}
	if timeout == 0 {
		timeout = 15 * time.Minute
	}

Related:

@maxcao13
UPSTREAM: <carry>: Extend HPA test timeout
c939878 
Signed-off-by: Max Cao <macao@redhat.com>
@openshift-ci-robot openshift-ci-robot added backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. labels Dec 4, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 4, 2025

@maxcao13: This pull request explicitly references no jira issue.

In response to this:

HPA tests are flaking because some are slow enough that they reach the 15 minute default timeout.
This PR sets the default test timeout of the HPA suite to 30 minutes to accommodate the slow tests.

HPA:

  // These tests take ~20 minutes each.
  f.Describe("ReplicationController", func() {
  	ginkgo.It(titleUp+" and verify decision stability", func(ctx context.Context) {
  		scaleUp(ctx, "rc", e2eautoscaling.KindRC, cpuResource, utilizationMetricType, true, f)
  	})
  	ginkgo.It(titleDown+" and verify decision stability", func(ctx context.Context) {
  		scaleDown(ctx, "rc", e2eautoscaling.KindRC, cpuResource, utilizationMetricType, true, f)
  	})
  })

openshift/origin:

  timeout := o.Timeout
  if timeout == 0 {
  	timeout = suite.TestTimeout
  }
  if timeout == 0 {
  	timeout = 15 * time.Minute
  }

Related:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 4, 2025

@maxcao13: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@maxcao13
Copy link
Member Author

maxcao13 commented Dec 4, 2025

/test ci/prow/k8s-e2e-aws-hpa

@openshift-ci
Copy link

openshift-ci bot commented Dec 4, 2025

@maxcao13: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test artifacts

/test configmap-scale

/test e2e-aws-crun-wasm

/test e2e-aws-jenkins

/test e2e-aws-ovn-cgroupsv2

/test e2e-aws-ovn-crun

/test e2e-aws-ovn-downgrade

/test e2e-aws-ovn-fips

/test e2e-aws-ovn-hypershift

/test e2e-aws-ovn-runc

/test e2e-aws-ovn-serial-1of2

/test e2e-aws-ovn-serial-2of2

/test e2e-aws-ovn-upgrade

/test e2e-azure-ovn-upgrade

/test e2e-gcp

/test e2e-gcp-ovn-upgrade

/test e2e-metal-ipi-ovn-ipv6

/test images

/test integration

/test k8s-e2e-conformance-aws

/test k8s-e2e-gcp-ovn

/test k8s-e2e-gcp-serial

/test okd-scos-images

/test perfscale-control-plane-6nodes

/test unit

/test verify

/test verify-commits

The following commands are available to trigger optional jobs:

/test e2e-agnostic-ovn-cmd

/test e2e-aws

/test e2e-aws-csi

/test e2e-aws-disruptive

/test e2e-aws-ovn

/test e2e-aws-ovn-techpreview

/test e2e-aws-ovn-techpreview-serial-1of2

/test e2e-aws-ovn-techpreview-serial-2of2

/test e2e-aws-single-node

/test e2e-azure

/test e2e-external-aws-ccm

/test e2e-metal-ipi-ovn-dualstack

/test e2e-openstack

/test e2e-openstack-csi-cinder

/test e2e-openstack-csi-manila

/test e2e-vsphere

/test k8s-e2e-aws

/test k8s-e2e-aws-ovn-serial

/test k8s-e2e-gcp-five-control-plane-replicas

/test okd-scos-e2e-aws-ovn

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-kubernetes-master-artifacts

pull-ci-openshift-kubernetes-master-e2e-aws-crun-wasm

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-cgroupsv2

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-crun

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-fips

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-hypershift

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-runc

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-serial-1of2

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-serial-2of2

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-techpreview

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-techpreview-serial-1of2

pull-ci-openshift-kubernetes-master-e2e-aws-ovn-techpreview-serial-2of2

pull-ci-openshift-kubernetes-master-e2e-gcp

pull-ci-openshift-kubernetes-master-e2e-metal-ipi-ovn-ipv6

pull-ci-openshift-kubernetes-master-images

pull-ci-openshift-kubernetes-master-integration

pull-ci-openshift-kubernetes-master-k8s-e2e-conformance-aws

pull-ci-openshift-kubernetes-master-k8s-e2e-gcp-ovn

pull-ci-openshift-kubernetes-master-k8s-e2e-gcp-serial

pull-ci-openshift-kubernetes-master-okd-scos-images

pull-ci-openshift-kubernetes-master-unit

pull-ci-openshift-kubernetes-master-verify

pull-ci-openshift-kubernetes-master-verify-commits

In response to this:

/test ci/prow/k8s-e2e-aws-hpa

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@maxcao13
Copy link
Member Author

maxcao13 commented Dec 4, 2025

/payload-job periodic-ci-openshift-kubernetes-master-k8s-e2e-aws-hpa

@openshift-ci
Copy link

openshift-ci bot commented Dec 4, 2025

@maxcao13: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-kubernetes-master-k8s-e2e-aws-hpa

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/604a3000-d157-11f0-8331-80e9a964f128-0

@maxcao13
Copy link
Member Author

maxcao13 commented Dec 5, 2025

/payload-job periodic-ci-openshift-kubernetes-master-k8s-e2e-aws-hpa

@openshift-ci
Copy link

openshift-ci bot commented Dec 5, 2025

@maxcao13: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-kubernetes-master-k8s-e2e-aws-hpa

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/fe2e5630-d16f-11f0-9934-d878749a4a0b-0

@maxcao13
Copy link
Member Author

maxcao13 commented Dec 5, 2025

/retest-required

@maxcao13 maxcao13 mentioned this pull request Dec 5, 2025
Merged
@openshift-ci
Copy link

openshift-ci bot commented Dec 5, 2025

@maxcao13: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@maxcao13
Copy link
Member Author

maxcao13 commented Dec 5, 2025

I don't know how to run the perioidic here but I've ran it on my own cluster with this change, and it works:

$ ./openshift-tests run kubernetes/autoscaling/hpa --dry-run --provider=aws --run="Should scale from 5 pods to 3 pods and then from 3 pods to 1 pod and verify decision stability"
# ...
# ...
started: 0/1/1 "[sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) ReplicationController Should scale from 5 pods to 3 pods and then from 3 pods to 1 pod and verify decision stability"

passed: (21m43s) 2025-12-05T23:00:35 "[sig-autoscaling] [Feature:HPA] Horizontal pod autoscaling (scale resource: CPU) ReplicationController Should scale from 5 pods to 3 pods and then from 3 pods to 1 pod and verify decision stability"

@maxcao13
Copy link
Member Author

maxcao13 commented Dec 5, 2025 •
edited
Loading

/cc @bertinatto
/assign @bertinatto

Can you take a look at this please :-) 🙏

bertinatto
bertinatto reviewed Dec 8, 2025
View reviewed changes
Copy link
Member

@bertinatto bertinatto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

This is a periodic job not part of payload testing:
/label acknowledge-critical-fixes-only

@openshift-ci openshift-ci bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Dec 8, 2025
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 8, 2025
@openshift-ci
Copy link

openshift-ci bot commented Dec 8, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bertinatto, maxcao13

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 8, 2025
@bertinatto
Copy link
Member

bertinatto commented Dec 8, 2025

/verified by @maxcao13

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Dec 8, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Dec 8, 2025

@bertinatto: This PR has been marked as verified by @maxcao13.

In response to this:

/verified by @maxcao13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@bertinatto
Copy link
Member

bertinatto commented Dec 8, 2025

/remove-label backports/unvalidated-commits

@openshift-ci openshift-ci bot removed the backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. label Dec 8, 2025
@bertinatto
Copy link
Member

bertinatto commented Dec 8, 2025 •
edited
Loading

/hold

Edit: we should be able to test this change by scheduling the periodic job with /payload-job

@openshift-ci openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 8, 2025
@bertinatto
Copy link
Member

bertinatto commented Dec 8, 2025

/payload-job periodic-ci-openshift-kubernetes-master-k8s-e2e-aws-hpa

@openshift-ci
Copy link

openshift-ci bot commented Dec 8, 2025

@bertinatto: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-kubernetes-master-k8s-e2e-aws-hpa

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/8a6b1080-d42b-11f0-9b71-46e3ac21ecbf-0

@maxcao13
Copy link
Member Author

maxcao13 commented Dec 8, 2025

@bertinatto The payload job command didn't work here either. Are you okay if I just unheld here?

@bertinatto
Copy link
Member

bertinatto commented Dec 8, 2025

/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 8, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit 05182ef into openshift:master Dec 8, 2025
24 checks passed
@maxcao13 maxcao13 deleted the hpa-timeout branch December 8, 2025 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bertinatto bertinatto bertinatto left review comments

@jerpeter1 jerpeter1 Awaiting requested review from jerpeter1

Assignees

@bertinatto bertinatto

Labels

acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@maxcao13 @openshift-ci-robot @bertinatto