Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax upper bound for dill #9419

Merged
merged 1 commit into from
Sep 3, 2019
Merged

Relax upper bound for dill #9419

merged 1 commit into from
Sep 3, 2019

Conversation

aaltay
Copy link
Member

@aaltay aaltay commented Aug 23, 2019

Dill now has a 0.3.0 version. Could not find any release notes to tell whether there are any breaking changes or not. (Asked here: uqfoundation/dill#223)


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- Build Status --- --- Build Status
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Build Status
Python Build Status
Build Status
Build Status
Build Status
--- Build Status
Build Status
Build Status --- --- Build Status
XLang --- --- --- Build Status --- --- ---

Pre-Commit Tests Status (on master branch)

--- Java Python Go Website
Non-portable Build Status Build Status Build Status Build Status
Portable --- Build Status --- ---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

Dill now has a 0.3.0 version. Could not find any release notes to tell whether there are any breaking changes or not. (Asked here: uqfoundation/dill#223)
@aaltay aaltay requested a review from robertwb August 23, 2019 17:53
@aaltay
Copy link
Member Author

aaltay commented Aug 23, 2019

Run Portable_Python PreCommit

@robertwb
Copy link
Contributor

The change looks fine, let's see if we get any response about the release notes.

@aaltay aaltay changed the title Release upper bound for dill Relax upper bound for dill Aug 23, 2019
@aaltay
Copy link
Member Author

aaltay commented Aug 26, 2019

Run Python 2 PostCommit

@aaltay
Copy link
Member Author

aaltay commented Aug 26, 2019

cc: @tvalentyn

@tvalentyn
Copy link
Contributor

Run Python 2 PostCommit

@tvalentyn
Copy link
Contributor

Run Python 3.5 PostCommit

@tvalentyn
Copy link
Contributor

Run Python 3.6 PostCommit

@tvalentyn
Copy link
Contributor

Run Python 3.7 PostCommit

@aaltay
Copy link
Member Author

aaltay commented Aug 27, 2019

Dataflow Jobs are failing with a strange error:

https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-08-26_16_09_21-13233476031481132526?project=apache-beam-testing

Workflow failed. Causes: Because of the shape of your pipeline, the Cloud Dataflow job optimizer produced a job graph that is not updatable using the --update pipeline option. This is a known issue that we are working to resolve. See https://issuetracker.google.com/issues/118375066 for information about how to modify the shape of your pipeline to avoid this error. You can override this error and force the submission of the job by specifying the --experiments=allow_non_updatable_job parameter., The stateful transform named 'WriteToBigQuery/BigQueryBatchFileLoads/ImpulseSingleElementPC/Decode Values.out/FromValue/ReadStream' is in two or more computations.

@aaltay
Copy link
Member Author

aaltay commented Aug 27, 2019

@tvalentyn could you take over this PR? I do not see the same issue in other postcommit tests. It could be a flake or somehow related to dill upgrade. I am not sure how.

/cc @angoenka since there is an streaming update related error here.

@Peque
Copy link

Peque commented Aug 27, 2019

@aaltay https://github.com/cloudpipe/cloudpickle has a changelog. 😉

@aaltay
Copy link
Member Author

aaltay commented Aug 27, 2019

Thank you @Peque . We are also considering switching to cloudpickle. (@tvalentyn is there an issue tracking that investigation?)

@tvalentyn
Copy link
Contributor

We started to consider a switch to cloudpickle along the sidelines of several issues, but it makes sense to dedicate an issue of its own to this migration, opened https://issues.apache.org/jira/browse/BEAM-8123.

@tvalentyn
Copy link
Contributor

Dataflow errors on this PR occurred in postcommit test suites last week, I think they are not specific to this change.

@tvalentyn
Copy link
Contributor

Run Python 2 PostCommit

@tvalentyn
Copy link
Contributor

Run Python 3.7 PostCommit

@aaltay
Copy link
Member Author

aaltay commented Aug 31, 2019

Run Python 3.5 PostCommit

@aaltay
Copy link
Member Author

aaltay commented Aug 31, 2019

Run Python 3.6 PostCommit

@aaltay
Copy link
Member Author

aaltay commented Aug 31, 2019

Run Python 3.5 PostCommit

@aaltay
Copy link
Member Author

aaltay commented Sep 1, 2019

Run Python 3.6 PostCommit

@tvalentyn tvalentyn merged commit dad64bf into master Sep 3, 2019
@aaltay aaltay deleted the aaltay-patch-2 branch September 3, 2019 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants