[SYSTEMDS-2747] Federated Als-CG Test#1170
Conversation
|
I've made a new commit (8c69017) to revert the changes of BinaryMatrixMatrixFEDInstruction.java because some federated algorithm tests were failing. In addition, I ignored the MatrixMatrix tests of the new FederatedLogicalTest (not working without the previous change) while still keeping the MatrixScalar tests (still working). |
…struction in checkAndReplaceSP() fix(BinaryMatrixMatrixFEDInstruction.java): change broadcast of mo2 to broadcast sliced feat(FedLogical): add tests for federated logical MatrixScalar and MatrixMatrix instructions fix(alsCG): add check for heavy hitter "fed_!=" - now supported for SPARK too :)
…dcasting sliced chore(fedLogical): ignore the MatrixMatrix tests - keep the MatrixScalar Tests
…t is a matrix and row partitioned distinguish the case where mo2 is a column vector --> don't broadcast slice even if it is row partioned chore(FedLogical): remove ignores of MatrixMatrix tests
ywcb00
force-pushed
the
test/fed/alscg
branch
from
8c69017 to
8059c5e
Compare
|
Found my mistake - we need to distinguish between column vector (normal broadcast) and matrix (broadcast sliced if row partitioned) in BinaryMatrixMatrixFEDInstruction.java. |
|
LGTM - thanks @ywcb00. I just disabled the spark tests for ALS because forced spark exec mode hurts here in terms of execution (repeated lazy evaluation as the no intermediates are materialized). As we discussed offline, please go ahead and do an additional cleanup over the federated binary element-wise operations to clearly differentiate between row and column partitioned federated data for matrix-row vector, matrix-column vector, and matrix-matrix. Thanks. |
2 participants
Hi,
This is a PR for adding federated Als-CG as a first algorithm-level test for federated quaternary operations.
I also added a branch inside FEDInstructionUtils for rewriting BinarySP instructions into BinaryFED instructions in order to support the "fed_!=" instruction needed inside the als-CG algorithm. Since I haven't found a test for these federated operations, I added a test for MatrixScalar and MatrixMatrix federated logical operations.
I made a small fix (I hope it was a fix xD) inside BinaryMatrixMatrixFEDInstruction by changing a broadcast() to a broadcastSliced().
Hope that was not too much, otherwise just revert the last commit (bce7fdc) and the AlsCG test is using "sp_!=" for spark execution mode again.
Thanks for review :)