Skip to content

[BEAM-2524] Update Dataflow FE URLs with regionalized paths.#3455

Closed
lostluck wants to merge 6 commits into
apache:masterfrom
lostluck:regionalize
Closed

[BEAM-2524] Update Dataflow FE URLs with regionalized paths.#3455
lostluck wants to merge 6 commits into
apache:masterfrom
lostluck:regionalize

Conversation

@lostluck
Copy link
Copy Markdown
Contributor

@lostluck lostluck commented Jun 27, 2017

Updates both the Java and the Python Dataflow Runners with the regionalized form of the Google Cloud Dataflow FE URL in anticipation of Regional Dataflow.

https://console.cloud.corp.google.com/dataflow/jobsDetail/locations/<regionId>/jobs/<jobId>?project=<projectId>

@lostluck
Copy link
Copy Markdown
Contributor Author

Hi @tgroh, @aaltay, can you please take a look?

@aaltay
Copy link
Copy Markdown
Member

aaltay commented Jun 27, 2017

Python changes LGTM .

There are lint errors (e.g. MonitoringUtil.java:187: Line is longer than 100 characters (found 114). [LineLength])

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.0007%) to 70.873% when pulling ac63dda on lostluck:regionalize into 0b19fb4 on apache:master.

}

public static String getJobMonitoringPageURL(String projectName, String jobId) {
public static String getJobMonitoringPageURL(String projectName, String regionId, String jobId) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is a backward incompatible change. Someone from the Java team should verify that this interface is not meant to be used by the end users.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to an expert verification.
I can certainly say that it's only being used outside the Dataflow runner, and certainly not the worker.
If all else fails, I can overload the method and sub in "us-central1" by default which should be sufficient to avoid breaking end user code.

Copy link
Copy Markdown
Member

@davorbonaci davorbonaci Jul 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do what you propose: "overload the method and sub in "us-central1" by default which should be sufficient to avoid breaking end user code"

(and deprecate the old one)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with davor; Simultaneously, we make no backwards-compatibility guarantees about this package.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@aaltay
Copy link
Copy Markdown
Member

aaltay commented Jun 29, 2017

Test failure is a flake I think.

@aaltay
Copy link
Copy Markdown
Member

aaltay commented Jun 29, 2017

jenkints retest this please

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.003%) to 70.87% when pulling ac63dda on lostluck:regionalize into 0b19fb4 on apache:master.

Copy link
Copy Markdown
Member

@tgroh tgroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Java LGTM, pending backwards-compatibility override.

}

public static String getJobMonitoringPageURL(String projectName, String jobId) {
public static String getJobMonitoringPageURL(String projectName, String regionId, String jobId) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with davor; Simultaneously, we make no backwards-compatibility guarantees about this package.

Use old getJobMonitoringPageURL signature as overload. Fill in "us-central1" as the default when it's used.
@lostluck
Copy link
Copy Markdown
Contributor Author

Added using the legacy path as an overload. PTAL

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.3%) to 70.622% when pulling 31cbd83 on lostluck:regionalize into 0b19fb4 on apache:master.

@@ -181,13 +181,18 @@ public List<JobMessage> getJobMessages(
}

public static String getJobMonitoringPageURL(String projectName, String jobId) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you tag the old one as Deprecated?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@lostluck
Copy link
Copy Markdown
Contributor Author

Added @deprecated annotation and @deprecated javadoc preferring use of the other overload. PTAL.

@aaltay
Copy link
Copy Markdown
Member

aaltay commented Jul 18, 2017

Thank you. LGTM. I will merge after tests pass.

@lostluck
Copy link
Copy Markdown
Contributor Author

Thanks!

@aaltay
Copy link
Copy Markdown
Member

aaltay commented Jul 18, 2017

There is a lint error [ERROR] /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_MavenInstall/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/util/MonitoringUtil.java:184: Line is longer than 100 characters (found 118). [LineLength]

@coveralls
Copy link
Copy Markdown

Coverage Status

Coverage decreased (-0.2%) to 70.635% when pulling a3a0c29 on lostluck:regionalize into 0b19fb4 on apache:master.

@asfgit asfgit closed this in dd9e866 Jul 18, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants