Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spanner: unflake TestIntegration_DbRemovalRecovery #1571

Closed
jeanbza opened this issue Sep 10, 2019 · 1 comment
Closed

Spanner: unflake TestIntegration_DbRemovalRecovery #1571

jeanbza opened this issue Sep 10, 2019 · 1 comment
Assignees
@olavloite
Labels
api: spanner Issues related to the Spanner API. type: cleanup An internal cleanup or hygiene concern.
Milestone
Tests clean up

Comments

@jeanbza
Copy link
Member

jeanbza commented Sep 10, 2019

https://source.cloud.google.com/results/invocations/d2cad416-31ba-41da-8cb1-2eac5a708b48/targets

https://sponge.corp.google.com/invocation?tab=Build+Log&id=d2cad416-31ba-41da-8cb1-2eac5a708b48&searchFor=

Failed
integration_test.go:893: failed to send query to database projects/dulcet-port-762/instances/go-integration-test/databases/gotest_1568136232283291228_09: spanner: code = "DeadlineExceeded", desc = "context deadline exceeded, lastErr is <rpc error: code = DeadlineExceeded desc = context deadline exceeded>"
integration_test.go:2317: failed to drop database projects/dulcet-port-762/instances/go-integration-test/databases/gotest_1568136232283291228_09 (error context deadline exceeded), might need a manual removal
@jeanbza jeanbza added api: spanner Issues related to the Spanner API. type: cleanup An internal cleanup or hygiene concern. labels Sep 10, 2019
@jeanbza jeanbza added this to the Tests clean up milestone Sep 10, 2019
gopherbot pushed a commit that referenced this issue Sep 16, 2019
@olavloite
spanner: Set timeout of all integration tests to 5 minutes
c981458 
Some integration tests had timeouts of 60 or 120 seconds,
others had no timeouts at all, while the majority had a
timeout of 5 minutes. All integration tests that create and
drop a test database have now been given a timeout of 5
minutes. This should reduce the chance that an integration
test will fail because of a timeout during database creation
or deletion.

Updates #1571.

Change-Id: I93283383789a75e009b5e9a1681e5e29ddf66517
Reviewed-on: https://code-review.googlesource.com/c/gocloud/+/45411
Reviewed-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jean de Klerk <deklerk@google.com>
gopherbot pushed a commit that referenced this issue Sep 17, 2019
@olavloite
spanner: Create and drop a test instance for each integration test run
67c9f1a 
Instead of using a fixed Spanner instance for all integration test runs,
this will create a new instance specifically for one test run and create
all test databases on the temporary instance. The temporary test instance
will be deleted after the test execution. The cleanup method will also
delete any old test instances that might have been left behind by a
previous test run, if these are more than 24 hours old.
Each integration test will create a new database, but will not drop it,
as the instance will be deleted at the end of the test run anyways. This
reduces the chance that a test case times out while dropping a database.

Updates #1571.

Change-Id: I27e369004c29b3ed1afe43b938db7e4e1afd23bc
Reviewed-on: https://code-review.googlesource.com/c/gocloud/+/45412
Reviewed-by: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jean de Klerk <deklerk@google.com>
@olavloite
Copy link
Contributor

This specific issue is no longer relevant, as we now create a new instance for each test run, and at the end of the test run delete the entire instance instead of individual databases. A test instance that fails to be deleted at the end of a test run, will be cleaned up automatically by subsequent test runs once they are more than 24 hours old (see

google-cloud-go/spanner/integration_test.go

Line 2481 in 20091f4

if instanceNameSpace.Older(inst.Name, expireAge) {
).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@olavloite olavloite

Labels
api: spanner Issues related to the Spanner API. type: cleanup An internal cleanup or hygiene concern.
Projects
None yet
Milestone
Tests clean up
Development

No branches or pull requests
2 participants
@olavloite @jeanbza