wreck: add support to set/get scheduler parameters at runtime #1579
Conversation
|
@dongahn: Changed the commit history/msg here as well (as per your suggestion earlier). |
Codecov Report
@@ Coverage Diff @@
## master #1579 +/- ##
==========================================
+ Coverage 79.14% 79.17% +0.02%
==========================================
Files 170 170
Lines 31182 31182
==========================================
+ Hits 24679 24688 +9
+ Misses 6503 6494 -9
|
|
Thanks @tpatki for putting this together. Generally LGTM! (With the caveat that my Lua isn't that great). The only thing I noticed is the merge commit. I believe the normal pattern for flux-framework repos is to rebase to master rather than merge with master, but @garlick or @grondo would know better than me. |
src/cmd/flux-wreck
Outdated
|
|
||
| for k,v in pairs(resp) do | ||
| result = tostring(k).."="..tostring(v) | ||
| print(result) |
There was a problem hiding this comment.
This would be extra work, but you could add support for flux wreck sched-params get ITEM here. As you iterate the response from the sched.params.get, check for key == ITEM and only print result if there is a match. As far as I'm concerned the current approach is fine for now, I only mean the comment as instructive.
src/cmd/flux-wreck
Outdated
| end | ||
|
|
||
| for k,v in pairs(resp) do | ||
| result = tostring(k).."="..tostring(v) |
There was a problem hiding this comment.
Not a big deal, but result should be declared local. Or better yet, just use print (k.."="..v), I think both k and v will be automatically promoted to strings, so the explicit use of tostring() is not needed.
| end | ||
|
|
||
| local resp, err = f:rpc ("sched.params.set", { param = tostring(params) }) | ||
| if not resp then |
There was a problem hiding this comment.
A better protocol would be for the JSON payload here to be a dictionary, e.g. instead of
{ "param": "item=42" }use
{ "item": 42 }However, looking at the sched project, I see this is because you are using an existing function that parses "item=value" at module load time.
If you end up having to refactor a bit in sched, I would suggest moving the rpc to the more flexible dictionary for the payload. If it is functional, then the current approach seems good enough for now.
|
Oh, yes, and please rebase onto master so we don't have the merge commit. Thanks! |
Not to go down a rat hole on this, but if you go that route, it may be a good idea for protocol extensibility to push the diciionary into its own object so that other non-dictionary keys can be added later (for example, a flags parameter): {
"dict":{
"item":42,
"foo":"bar",
},
"flags":0,
}but as @grondo said, what you have makes sense for now. |
|
@grondo: rebased, and made the change you requested for printing individual parameters. I'm working on refactoring the @grondo, @garlick: I will open another issue to refactor the sched end to support the dictionary payload you both suggested. |
|
@dongahn, @SteVwonder, @grondo: |
garlick
changed the title
|
Thanks @tpatki! Will try to do a quick test early today, but looks good to me. |
|
Ok, did some quick testing and things look good! One thing I noticed is that |
|
e.g.: (flux-69360) grondo@ipa15:~/git/flux-sched.git$ flux wreck sched-params set queue-depth=-1024
(flux-69360) grondo@ipa15:~/git/flux-sched.git$ flux wreck sched-params get
delay-sched=false
queue-depth=-1024Jobs don't seem to run with a negative queue-depth. |
|
Actually the fixes will likely be on the sched side since this is just client code. Merging. |
|
@grondo: Thanks! Yes, we need some error checking around queue_depth, I'll open another issue for this. |
Support for sched/issue337.
Add
flux wreck sched-params set/getoptions to support getting/setting runtime parameters in sched.flux wreck sched-params get,flux wreck sched-params get queue-depth, and `flux-wreck sched-params get queue-depth,delay-sched' should all work.