Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark/JNI multipass fixes and AllReduce quiet support #1949

Merged
merged 39 commits into from Jun 27, 2019

Conversation

Projects
None yet
3 participants
@eisber
Copy link
Collaborator

commented Jun 25, 2019

No description provided.

vagrant and others added some commits Mar 9, 2019

eisber and others added some commits May 15, 2019

clean up of CMakeFile.txt
formatted code
improved Spanning Tree error message
re-added README.md
moved JNI util to the right place
fixed cache file support by passing --no_stdin
added quiet for AllReduce
replaced cerr with THROW for AllReduce
@@ -287,7 +338,7 @@ void AllReduceSockets::broadcast(char* buffer, const size_t n)
int read_size = recv(socks.parent, buffer + parent_read_pos, (int)count, 0);
if (read_size == -1)
{
cerr << " recv from parent: " << strerror(errno) << endl;
THROW(" recv from parent: " << strerror(errno));

This comment has been minimized.

Copy link
@jackgerrits

jackgerrits Jun 26, 2019

Member

Why change these to throw?

This comment has been minimized.

Copy link
@eisber

eisber Jun 26, 2019

Author Collaborator

in reality any silent failure will cause my spark application to hang. throwing will bubble things up

@eisber eisber merged commit 855f0c5 into VowpalWabbit:master Jun 27, 2019

6 of 8 checks passed

LGTM analysis: Java Analysis failed (could not build the merge commit and the base commit (5d1527c))
Details
coverage/coveralls Coverage decreased (-0.05%) to 73.35%
Details
LGTM analysis: C# No code changes detected
Details
LGTM analysis: C/C++ No new or fixed alerts
Details
LGTM analysis: JavaScript No code changes detected
Details
LGTM analysis: Python No code changes detected
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.