Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase provisioning timeouts #229

Merged
merged 1 commit into from
Aug 13, 2022

Conversation

mfielding
Copy link
Contributor

To help deal with slow underlying storage, we need to adjust a few timeouts:

  • bootstrapTimeout in init_oracle.go: Instance bootstrap involves file moves, three instance restarts and two Oracle resetlogs operations, all of which involve disk I/O. Here we increase the timeout from 19 to 29 minutes.
  • DatabaseInstanceReadyTimeoutSeeded in instance_controller.go: this timeout is set at the same time as bootstrapTimeout, but in the El Carro operator rather than in the destination database container. We set it to 1 minute more than bootstrapTimeout.
  • InstanceReadyTimeout in instance_controller.go: this timeout covers the actual creation of the pod, and notably provisioning any storage dependencies. This large increase is designed to cover storage platforms that provision multiple volumes (Data, Log, Backup) serially.
  • CallTimeout in dbdaemonlib.go: this timeout refers to dbdaemon calls after initial instance provisioning completes, notably CreateDatabase which creates an Oracle pluggable database, and again involves disk I/O. This change mirrors PR Increase default timeout for dbdaemon calls #227 , added here to hopefully make merging easier.

To help deal with slow underlying storage, we need to adjust a few timeouts:

bootstrapTimeout in init_oracle.go:  Instance bootstrap involves file moves,
three instance restarts and two Oracle "resetlogs" operations, all of which
involve disk I/O.  Here we increase the timeout from 19 to 29 minutes.

* DatabaseInstanceReadyTimeoutSeeded in instance_controller.go: this timeout
  is set at the same time as bootstrapTimeout, but in the El Carro operator
  rather than in the destination database container.  We set it to 1 minute
  more than bootstrapTimeout.
* InstanceReadyTimeout in instance_controller.go:  this timeout covers the
  actual creation of the pod, and notably provisioning any storage
  dependencies.  This large increase is designed to cover storage platforms
  that provision multiple volumes (Data, Log, Backup) serially.
* CallTimeout in dbdaemonlib.go:  this timeout refers to dbdaemon calls
  after initial instance provisioning completes, notably CreateDatabase
  which creates an Oracle pluggable database, and again involves disk I/O.
  This change mirrors PR # 227, added here to hopefully make merging easier.
Copy link
Collaborator

@akinfermo akinfermo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akinfermo, mfielding

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@akinfermo akinfermo merged commit b440f3a into GoogleCloudPlatform:main Aug 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants