Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark/JNI multipass fixes and AllReduce quiet support #1949

merged 39 commits into from Jun 27, 2019


None yet
3 participants
Copy link

commented Jun 25, 2019

No description provided.

vagrant and others added some commits Mar 9, 2019

eisber and others added some commits May 15, 2019

clean up of CMakeFile.txt
formatted code
improved Spanning Tree error message
moved JNI util to the right place
fixed cache file support by passing --no_stdin
added quiet for AllReduce
replaced cerr with THROW for AllReduce
@@ -287,7 +338,7 @@ void AllReduceSockets::broadcast(char* buffer, const size_t n)
int read_size = recv(socks.parent, buffer + parent_read_pos, (int)count, 0);
if (read_size == -1)
cerr << " recv from parent: " << strerror(errno) << endl;
THROW(" recv from parent: " << strerror(errno));

This comment has been minimized.

Copy link

jackgerrits Jun 26, 2019


Why change these to throw?

This comment has been minimized.

Copy link

eisber Jun 26, 2019

Author Collaborator

in reality any silent failure will cause my spark application to hang. throwing will bubble things up

@eisber eisber merged commit 855f0c5 into VowpalWabbit:master Jun 27, 2019

6 of 8 checks passed

LGTM analysis: Java Analysis failed (could not build the merge commit and the base commit (5d1527c))
coverage/coveralls Coverage decreased (-0.05%) to 73.35%
LGTM analysis: C# No code changes detected
LGTM analysis: C/C++ No new or fixed alerts
LGTM analysis: JavaScript No code changes detected
LGTM analysis: Python No code changes detected
continuous-integration/appveyor/pr AppVeyor build succeeded
continuous-integration/travis-ci/pr The Travis CI build passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.