Skip to content

[BEAM-8472] Get default GCP region from gcloud (Java)#9974

Merged
ibzib merged 1 commit intoapache:masterfrom
ibzib:java-default-region
Nov 8, 2019
Merged

[BEAM-8472] Get default GCP region from gcloud (Java)#9974
ibzib merged 1 commit intoapache:masterfrom
ibzib:java-default-region

Conversation

@ibzib
Copy link

@ibzib ibzib commented Nov 1, 2019

Same as #9868 but more verbose 😄

As with the Python version, I've verified this works locally. This doesn't fit into a unit test because it requires special environment configuration and/or calling an external process, but nor does it seem to warrant an integration test.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- Build Status --- --- Build Status
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status
--- Build Status
Build Status
Build Status
Build Status
--- --- Build Status
XLang --- --- --- Build Status --- --- ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status
Build Status
Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@ibzib ibzib requested a review from aaltay November 1, 2019 23:54
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Is 1 second enough?
  • Should we check the return value of this call? (IIUC it returns false for timeout.)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is 1 second enough?

Tests on my machine (in debug mode) indicated it took on average around 0.25 seconds, including reading the output stream after. I have bumped the timeout to 2s just to make double sure.

Should we check the return value of this call?

Not sure it matters, since I think it should interrupt the process after the time limit has passed, but it couldn't hurt to check anyway.

@ibzib ibzib force-pushed the java-default-region branch 2 times, most recently from 5102f46 to c46c2e8 Compare November 4, 2019 23:43
@ibzib
Copy link
Author

ibzib commented Nov 5, 2019

Run Java PreCommit

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason why we aren't using the default instance factory to configure the region instead of detecting and setting it here. This would mean that all users of the pipeline option would get it and not just Dataflow.

See DefaultProjectFactory for an example.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made a default instance factory.

@ibzib ibzib force-pushed the java-default-region branch from c46c2e8 to 8b005ed Compare November 5, 2019 18:50
@ibzib ibzib force-pushed the java-default-region branch from 8b005ed to 8e49c5a Compare November 5, 2019 18:54
@ibzib ibzib merged commit df0e7e8 into apache:master Nov 8, 2019
@Test
public void testDefaultGcpRegion() {
DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
assertEquals("us-central1", options.getRegion());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test will fail on any machine where the CLOUDSDK_COMPUTE_REGION is defined or gcloud config compute/region is set.

You can test your code by instantiating the DefaultGcpRegionFactory directly and ensuring that it is factored in such a way where you can pass in the environment map directly to it and similarly for testing the process execution.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this Luke. Filed #10048 to improve this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants