Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(spanner): long running transaction clean up - disabled #8177

Conversation

harshachinta
Copy link
Contributor

@harshachinta harshachinta commented Jun 27, 2023

Automatically cleaning long running transactions

@harshachinta harshachinta requested review from a team as code owners June 27, 2023 05:11
@product-auto-label product-auto-label bot added size: l Pull request size is large. api: spanner Issues related to the Spanner API. labels Jun 27, 2023
@harshachinta harshachinta changed the title feat(spanner): long running transaction clean up to prevent session leaks feat(spanner): long running transaction clean up Jun 27, 2023
@harshachinta harshachinta added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jun 27, 2023
@harshachinta harshachinta requested review from olavloite and removed request for olavloite July 4, 2023 03:49
spanner/client_test.go Outdated Show resolved Hide resolved
spanner/client_test.go Outdated Show resolved Hide resolved
spanner/client_test.go Outdated Show resolved Hide resolved
spanner/session.go Outdated Show resolved Hide resolved
spanner/session.go Outdated Show resolved Hide resolved
spanner/session.go Outdated Show resolved Hide resolved
spanner/session.go Outdated Show resolved Hide resolved
…le function to avoid sleep statements during unit tests
…dules. Currently tests are not getting run as part of github presubmits.
internal/kokoro/presubmit.sh Outdated Show resolved Hide resolved
spanner/client.go Show resolved Hide resolved
spanner/client.go Show resolved Hide resolved
spanner/client.go Outdated Show resolved Hide resolved
spanner/pdml.go Show resolved Hide resolved
spanner/transaction.go Outdated Show resolved Hide resolved
spanner/session.go Show resolved Hide resolved
spanner/session.go Show resolved Hide resolved
spanner/session.go Outdated Show resolved Hide resolved
spanner/transaction.go Outdated Show resolved Hide resolved
@product-auto-label product-auto-label bot added the stale: old Pull request is old and needs attention. label Jul 27, 2023
@product-auto-label product-auto-label bot added stale: extraold Pull request is critically old and needs prioritization. and removed stale: old Pull request is old and needs attention. labels Aug 26, 2023
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Oct 6, 2023
@@ -31,4 +31,87 @@ row, err := client.Single().ReadRow(ctx, "Users",
if err != nil {
log.Fatal(err)
}
```
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arpan14 @olavloite @rahul2393 @asthamohta
Can you please take a look at the golang documentation for session leaks and give your LGTM?

spanner/README.md Outdated Show resolved Hide resolved
The most common reason for session leaks in the Golang client library are:
1. Not stopping a `RowIterator` that is returned by `Query`, `Read` and other methods. Always use `RowIterator.Stop()` to ensure that the `RowIterator` is always closed.
2. Not closing a `ReadOnlyTransaction` when you no longer need it. Always call `ReadOnlyTransaction.Close()` after use, to ensure that the `ReadOnlyTransaction` is always closed.
3. Not closing a `BatchReadOnlyTransaction` when you no longer need it. Always call `BatchReadOnlyTransaction.Close()` after use, to ensure that the `BatchReadOnlyTransaction` is always closed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a different PR: We should change the implementation of BatchReadOnlyTransaction to not use any sessions from the pool, and instead create a separate session.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BatchReadOnlyTransaction already creates its own session and do not use any sessions from the pool. Sorry for the confusion.
I have removed this line.

spanner/README.md Outdated Show resolved Hide resolved
spanner/README.md Outdated Show resolved Hide resolved
spanner/README.md Outdated Show resolved Hide resolved
spanner/README.md Outdated Show resolved Hide resolved
spanner/README.md Outdated Show resolved Hide resolved
spanner/README.md Outdated Show resolved Hide resolved
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: xl Pull request size is extra large. labels Oct 6, 2023
@harshachinta harshachinta changed the title feat(spanner): long running transaction clean up feat(spanner): long running transaction clean up - disabled Oct 30, 2023
@harshachinta harshachinta removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Oct 30, 2023
@harshachinta harshachinta merged commit 461d11e into googleapis:main Oct 30, 2023
10 checks passed
bhshkh pushed a commit that referenced this pull request Nov 3, 2023
* feat(spanner): long running transaction clean up to prevent session leaks

* feat(spanner): code refactoring in session.go file

* fix(spanner): fix vet

* feat(spanner): refactor client.go file

* feat(spanner): add lock when updating session handle

* feat(spanner): code refactoring in transaction.go file

* test(spanner): remove test

* feat(spanner): code refactor

* feat(spanner): refactor nit comments

* feat(spanner): reduce sleep timing to milli seconds for unit tests

* feat(spanner): update idleTimeThresholdSecs field to time.Duration

* feat(spanner): make the log messages conditional based on type of action for inactive transactions

* feat(spanner): combine get and remove long running sessions in a single function to avoid sleep statements during unit tests

* feat(spanner): modify presubmit condition to run tests for changed modules. Currently tests are not getting run as part of github presubmits.

* feat(spanner): revert presubmit.sh fix

* feat(spanner): update doc

* feat(spanner): reword isLongRunning to eligibleForLongRunning in sessionHandle

* feat(spanner): update TrackSessionHandles logic to turn off stack trace by default for long running sessions

* feat(spanner): fix test

* feat(spanner): change action on inactive transactions option to enum

* feat(spanner): fix lint

* feat(spanner): fix lint

* feat(spanner): fix lint

* feat(spanner): fix doc

* feat(spanner): disable the feature by default

* feat(spanner): revert commit - disable the feature by default

* fix: lint

* docs: add Readme

* feat(spanner): make WARN as default for action on inactive transactions

* docs: address review comments

* docs: move README.md changes in a different PR

* feat: disable feature
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the Spanner API. size: l Pull request size is large. stale: extraold Pull request is critically old and needs prioritization.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants