fix: Fix BigtableIO to share one BigtableService per worker and increase default attempt timeout

[mutianf]

fix #26166

In #24015, the map that stores BigtableService is transient. During serialized and deserialized, it recreates an empty map which ended up creating a new BigtableService per each thread. The fix makes the map static.

In the previous version, mutateRows operation has a timeout of 6 minutes. And after the migration, the default timeout became 1 minute, which makes the job fail more easily. Update the timeout back to 6 minutes.

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[github-actions]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @robertwb for label java.
R: @ahmedabu98 for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[jrmccluskey]

Run Java_GCP_IO_Direct PreCommit

[Abacn]

Run Java_GCP_IO_Direct PreCommit

[jrmccluskey]

It looks like there's a somewhat consistent timeout in testTriggeredFileLoadsWithTempTablesToExistingNullSchemaTable:

17:37:22 org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOWriteTest > testTriggeredFileLoadsWithTempTablesToExistingNullSchemaTable[1] FAILED
17:37:22     org.junit.runners.model.TestTimedOutException at BigQueryIOWriteTest.java:729
17:45:55 
17:45:55 1645 tests completed, 1 failed, 124 skipped

Is this a matter of needing higher bounds on the direct runner?

[Abacn]

@jrmccluskey This is known issue #25207. It was causing the whole GCP_IO test suite time out before I added test timeout of 10min per test. From the log the suspected cause is some racing condition that temp file gets deleted when the test run and then test hanging. It happens occassionally but recently becomes more frequent. Possibly because other tests get added to GCP_IO then the order that junit running these test changed.

[jrmccluskey]

retest this please

[jrmccluskey]

Run Java_Examples_Dataflow_Java17 PreCommit

[jrmccluskey]

Run Java_GCP_IO_Direct PreCommit

[jrmccluskey]

Hmm still getting that Spanner IO failure. I don't have reason to believe it's related to this PR though. CC: @johnjcasey

[Abacn]

Hmm still getting that Spanner IO failure. I don't have reason to believe it's related to this PR though. CC: @johnjcasey

That is #26208 unrelated to Beam

[jrmccluskey]

Ah noted. In that case I'm comfortable merging this, the Java 11 examples passed but the PR didn't get updated