Skip to content

[MINOR] RmVar disable for federated read data.#1110

Merged
Baunsgaard merged 1 commit intoapache:masterfrom
Baunsgaard:master
Nov 20, 2020
Merged

[MINOR] RmVar disable for federated read data.#1110
Baunsgaard merged 1 commit intoapache:masterfrom
Baunsgaard:master

Conversation

@Baunsgaard
Copy link
Copy Markdown
Contributor

No description provided.

@Baunsgaard
Copy link
Copy Markdown
Contributor Author

Before
  2  tak+*         16.672   1200
After
  2  tak+*          6.257    600

@mboehm7
Copy link
Copy Markdown
Contributor

mboehm7 commented Nov 19, 2020

adding ternary aggregates is fine, but has nothing to do with the title of disabling rmvar right? Regarding the deleted wait on cleanup, please be aware that this might require federated worker changes as now they have to deal with concurrent RPC on a single execution context (not just concurrent RPC on different execution contexts that are managed in a concurrent hashmap). So it has a chance of brittle failures when these RPCs conflict on symbol table updates.

@Baunsgaard
Copy link
Copy Markdown
Contributor Author

Yes, I have realized that, i have simply gone back to having a background thread doing the cleanup.
Olga added a line at some point, forcing this to be run in the same thread, this made the rm-var a blocking operation on federated variables, unlike before.

@Baunsgaard
Copy link
Copy Markdown
Contributor Author

adding ternary aggregates is fine, but has nothing to do with the title of disabling rmvar right? Regarding the deleted wait on cleanup, please be aware that this might require federated worker changes as now they have to deal with concurrent RPC on a single execution context (not just concurrent RPC on different execution contexts that are managed in a concurrent hashmap). So it has a chance of brittle failures when these RPCs conflict on symbol table updates.

no ternary has nothing to do with the pr.. but it was one of the operation s making L2SVM slow, so i just added it.
Unfortunally it is only relevant if y is federated as well. But in that case, the wide network becomes very slow.

This commit adds Federated Ternary Aggregation, as well as change the
federated cleanup, to run in separated threads. The latter improve the
system performance by synchronizing and cleaning workers in parallel with
computation continuing.

Also a minior syntax error is corrected in builtin l2svm.

closes apache#1110
@Baunsgaard Baunsgaard merged commit ddc5dec into apache:master Nov 20, 2020
@github-pages github-pages bot temporarily deployed to github-pages November 20, 2020 13:10 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants