You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
cmd/compile: k8s scalability load test failure (latency regression) with unified IR #54593
Just to confirm, this is an issue with run-time code being slower? Like it's not a problem with the compiler having slowed down too much?
One thing that comes to mind is inlining has changed. In particular, the generated IR is somewhat different now, whereas the inlining heuristics are still tuned for the non-unified frontend's IR; and unified IR still has a more primitive heuristic for when to re-export function bodies, so sometimes inlining may fail. And in turn, this might have impacts on escape analysis.
Nope, it's about code at runtime. Now that I'm looking at some of the graphs, it actually looks more like occasional stalls than a uniform slowdown, though I think there is some smaller uniform impact on tail latencies as well in between those stalls (unclear if they're related yet).
Here's what the API call latency looks like over time (note the threshold for the first graph is 1 second):
And here's some GC-related statistics (note the breaks in the graphs that line up with the spikes):
Activity
mknyszek commentedon Aug 22, 2022
Assigning to myself for now to gather more information.
mdempsky commentedon Aug 22, 2022
Just to confirm, this is an issue with run-time code being slower? Like it's not a problem with the compiler having slowed down too much?
One thing that comes to mind is inlining has changed. In particular, the generated IR is somewhat different now, whereas the inlining heuristics are still tuned for the non-unified frontend's IR; and unified IR still has a more primitive heuristic for when to re-export function bodies, so sometimes inlining may fail. And in turn, this might have impacts on escape analysis.
mknyszek commentedon Aug 22, 2022
Nope, it's about code at runtime. Now that I'm looking at some of the graphs, it actually looks more like occasional stalls than a uniform slowdown, though I think there is some smaller uniform impact on tail latencies as well in between those stalls (unclear if they're related yet).
Here's what the API call latency looks like over time (note the threshold for the first graph is 1 second):
And here's some GC-related statistics (note the breaks in the graphs that line up with the spikes):
mdempsky commentedon Aug 30, 2022
Kubernetes reports this is fixed since 6a801d3.