[BEAM-2143]: Fix default temp location for DataflowRunner #2907
Conversation
|
|
||
| @Override | ||
| public String create(PipelineOptions options) { | ||
| GcsOptions gcsOptions = options.as(GcsOptions.class); | ||
| LOG.info("No staging location provided, falling back to temp location."); |
There was a problem hiding this comment.
suggest formatting the options as options: "No stagingLocation provided, falling back to gcpTempLocation"
| @@ -278,7 +278,7 @@ static String tryCreateDefaultBucket( | |||
| } | |||
| final String bucketName = | |||
| "dataflow-staging-" + region + "-" + projectNumber; | |||
| LOG.info("No staging location provided, attempting to use default bucket: {}", | |||
| LOG.info("No temp location provided, attempting to use default bucket: {}", | |||
There was a problem hiding this comment.
"No tempLocation specified, attempting to use default bucket:"
|
If If both |
|
Done |
|
The exception I get from this is still not particularly useful. It says: |
|
What command did you run? You shouldn't see an exception after this change for missing either of |
|
Ah -- it looks like it wasn't using the patched version of the SDK. Looks good now. |
|
@bjchambers @davorbonaci Looks good to be merged. |
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
[BEAM-<Jira issue #>] Description of pull requestmvn clean verify.<Jira issue #>in the title with the actual Jira issuenumber, if there is one.
Individual Contributor License Agreement.
gs://some-bucketpath invalid and expects it to be of the formgs://some-bucket/basename. So adding/tempas the basename for the defaulttempLocation.