Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: parallel integration tests #22

Merged
merged 30 commits into from
Aug 19, 2021

Conversation

olavloite
Copy link
Collaborator

@olavloite olavloite commented Aug 9, 2021

Makes all integration tests run in parallel by letting each test use its own database.
This also enables the integration tests to be executed in parallel on the emulator, as
the emulator does allow multiple operations and transactions to be executed in parallel
as long as these are on different databases.

Also adds a GitHub Actions configuration file to execute integration tests on the
emulator during presubmits.

#20 should be reviewed and merged before this.

Fixes #21

Makes all integration tests run in parallel by letting each test use its own database.
This also enables the integration tests to be executed in parallel on the emulator, as
the emulator does allow multiple operations and transactions to be executed in parallel
as long as these are on different databases.

Also adds a GitHub Actions configuration file to execute integration tests on the
emulator during presubmits.

Fixes googleapis#21
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Aug 9, 2021
@olavloite olavloite marked this pull request as ready for review August 11, 2021 09:55
@@ -0,0 +1,717 @@
// Copyright 2021 Google Inc. All Rights Reserved.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be:

// Copyright 2021 Google LLC

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for all other files.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do that in a separate update to prevent too many merge conflicts. As you mentioned, this should be changed in all the files, including files that are not included in this change.

keyValue := strings.SplitN(keyValueString, "=", 2)
if keyValue == nil || len(keyValue) != 2 {
return nil, fmt.Errorf("invalid connection property: %s", keyValueString)
}
params[keyValue[0]] = keyValue[1]
params[strings.ToLower(keyValue[0])] = keyValue[1]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this case insensitive?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sure that any additional parameters in the connection string are case insensitive. So a user does not have to worry whether the parameter name is for example usePlainText or useplaintext (that parameter name would normally be clear for everyone, but for parameters like readonly vs readOnly and autocommit vs autoCommit it's easy to make a mistake).

integration_test.go Outdated Show resolved Hide resolved
transaction.go Outdated Show resolved Hide resolved
transaction.go Outdated
type retriableStatement interface {
// retry retries the statement on a new Spanner transaction. The method must
// return nil if it receives the same result as during the initial attempt,
// and otherwise ErrAbortedDueToConcurrentModification.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise throws an ErrAbortedDueToConcurrentModification error.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Go the error is returned and not thrown, so I changed it to and otherwise return the error ErrAbortedDueToConcurrentModification. (Note: 'an ErrAbortedDueToConcurrentModification' is not used here, as this is a global error.)

return err
}

// Commit implements driver.Tx#Commit().
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move the second line in the comment to the first line?

// Commit implements driver.Tx#Commit(). It will commit the underlying ...
...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The godoc convention that the first sentence should start with the name of the method, so that would not work with that.

@@ -85,10 +233,59 @@ func (tx *readWriteTransaction) Rollback() error {
return nil
}

func (tx *readWriteTransaction) Query(ctx context.Context, stmt spanner.Statement) *spanner.RowIterator {
return tx.rwTx.Query(ctx, stmt)
func (tx *readWriteTransaction) Query(ctx context.Context, stmt spanner.Statement) rowIterator {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a comment to Query.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Collaborator

@hengfengli hengfengli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Left a few comments.

@hengfengli
Copy link
Collaborator

@olavloite Can you please rebase other following PRs after this is merged? Thanks.

@olavloite olavloite merged commit cf5d570 into googleapis:main Aug 19, 2021
@olavloite olavloite deleted the parallel-integration-tests branch August 19, 2021 09:49
This was referenced Aug 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parallelize integration tests
2 participants