-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Combine pmp2sdp and sdpb into a single executable? #78
Comments
SDPB restart
We can run SDPB with the following options:
Behavior:
NB: This can be problematic in the following scenario:
Possible solutions:
Thus, if we talk only about usability (and not about IO performance), maybe it's still better to keep two separate executables. |
Distributing PMP matrices and SDP blocksSpeaking of performance, there is a problem with distributing the blocks among the cores. Current behavior:In sdp2input, each core stores and processes only some polynomial matrices, according to a simple rule
In SDPB, we distribute blocks among cores according to block costs (which are read from timing data or estimated by block sizes) Moreover, a single block can be stored as a The problemIf we want to keep everything in-memory (without writing and reading If timing data is available, we can use it from the very beginning. If there is no timing data, then we can read PMP matrices as we do now, and then perform some non-trivial MPI messaging to redistribute them. Maybe it will not be significantly faster than just writing to disk and reading again. Anyway, all this requires non-trivial code changes, and we should do it only if IO for |
Problem
Common use case:
These intermediate IO operations can be quite expensive (tens of minutes). In Skydive, sdp2input+sdpb are called for each iteration, and IO takes up to ~50% of total sdpb time (as noted by @suning-git).
Solution
Create a single executable that accepts input in different formats and performs in-memory conversion to the format accepted by solver.
Potential issues
See comments below for details.
sdp.zip
is needed for SDPB restart. SDPB has to decide whether to overwritesdp.zip
or ignore PMP input, if both are present. May lead to subtle bugs if the user doesn't really understand SDPB behavior.The text was updated successfully, but these errors were encountered: