Please Support Arbitrary Labels and Gotos. #796
Comments
oridb
changed the title
|
@oridb Wasm is somewhat optimized for the consumer to be able to quickly convert to SSA form, and the structure does help here for common code patterns, so the structure is not necessarily a burden for the consumer. I disagree with your assertion that 'both sides of the code generation work around the format specified'. Wasm is very much about a slim and fast consumer, and if you have some proposals to make it slimmer and faster then that might be constructive. Blocks that can be ordered into a DAG can be expressed in the wasm blocks and branches, such as your example. The switch-loop is the style used when necessary, and perhaps consumers might do some jump threading to help here. Perhaps have a look at binaryen which might do much of the work for your compiler backend. There have been other requests for more general CFG support, and some other approaches using loops mentioned, but perhaps the focus is elsewhere at present. I don't think there are any plans to support 'continuation passing style' explicitly in the encoding, but there has been mention of blocks and loops popping arguments (just like a lambda) and supporting multiple values (multiple lambda arguments) and adding a |
I'm not seeing any common code pattern that are easier to represent in terms of branches to arbitrary labels, vs the restricted loops and blocks subset that web assembly enforces. I could see a minor benefit if there was an attempt to make the code closely resemble the input code for certain classes of langauge, but that doesn't seem to be a goal -- and the constructs are a bit bare if they were there for
Yes, they can be. However, I'd strongly prefer not to add extra work to determine which ones can be represented this way, versus which ones need extra work. Realistically, I'd skip doing the extra analysis, and always just generate the switch loop form. Again, my argument isn't that loops and blocks make things impossible; It's that everything they can do is simpler and easier for a machine to write with goto, goto_if, and arbitrary, unstructured labels.
I already have a serviceable backend that I'm fairly happy with, and plans to fully bootstrap the entire compiler in my own language. I'd rather not add in a rather large extra dependency simply to work around the enforced use of loops/blocks. If I simply use switch loops, emitting the code is pretty trivial. If I try to actually use the features present in web assembly effectively, instead of doing my damndest to pretend they don't exist, it becomes a good deal more unpleasant.
I'm still not convinced that loops have any benefits -- anything that can be represented with a loop can be represented with a goto and label, and there are fast and well known conversions to SSA from flat instruction lists. As afar as CPS goes, I don't think that there needs to be explicit support -- it's popular in FP circles because it's fairly easy to convert to assembly directly, and gives similar benefits to SSA in terms of reasoning (http://mlton.org/pipermail/mlton/2003-January/023054.html); Again, I'm not an expert on it, but from what I remember, the invocation continuation gets lowered to a label, a few movs, and a goto. |
Would be interesting to know how they compare with wasm SSA decoders, that is the important question? Wasm makes use of a values stack at present, and some of the benefits of that would gone without the structure, it would hurt decoder performance. Without the values stack the SSA decoding would have more work too, I've tried a register base code and decoding was slower (not sure how significant that is). Would you keep the values stack, or use a register based design? If keeping the values stack then perhaps it becomes a CIL clone, and perhaps wasm performance could be compared to CIL, has anyone actually check this? |
I don't actually have any strong feelings on that end. I'd imagine compactness of the encoding would be one of the biggest concerns; A register design may not fare that well there -- or it may turn out to compress fantastically over gzip. I don't actually know off the top of my head. Performance is another concern, although I suspect that it might be less important given the ability to cache binary output, plus the fact that download time may outweigh the decoding by orders of magnitude.
If you're decoding to SSA, that implies that you'd also be doing a reasonable amount of optimization. I'd be curious to benchmark how significant decoding performance is in the first place. But, yes, that's definitely a good question. |
|
Thanks for your questions and concerns. It's worth noting that many of the designers and implementors of The design of WebAssembly's control flow constructs simplifies consumers by We've had a lot of internal discussion between members about this very On Thu, Sep 8, 2016 at 9:53 AM, Ori Bernstein notifications@github.com
|
|
Thanks @titzer, I was developing a suspicion that Wasm's structure had a purpose beyond just similarity to asm.js. I wonder though: Java bytecode (and CIL) don't model CFGs or the value stack directly, they have to be inferred by the JIT. But in Wasm (especially if block signatures are added) the JIT can easily figure out what's going on with the value stack and control flow, so I wonder, if CFGs (or irreducible control flow specifically) were modeled explicitly like loops and blocks are, might that avoid most of the nasty corner cases you're thinking of? There's this neat optimization that interpreters use that relies on irreducible control flow to improve branch prediction... |
I agree that gotos are very useful for many compilers. That's why tools like Binaryen let you generate arbitrary CFGs with gotos, and they can convert that very quickly and efficiently into WebAssembly for you. It might help to think of WebAssembly as a thing optimized for browsers to consume (as @titzer pointed out). Most compilers should probably not generate WebAssembly directly, but rather use a tool like Binaryen, so that they can emit gotos, get a bunch of optimizations for free, and don't need to think about low-level binary format details of WebAssembly (instead you emit an IR using a simple API). Regarding polyfilling with the while-switch pattern you mention: in emscripten we started out that way before we developed the "relooper" method of recreating loops. The while-switch pattern is around 4x slower on average (but in some cases significantly less or more, e.g. small loops are more sensitive). I agree with you that in theory jump-threading optimizations could speed that up, but performance will be less predictable as some VMs will do it better than others. It is also significantly larger in terms of code size. |
I'm still not convinced that this aspect is going to matter that much - again, I suspect the cost of fetching the bytecode would dominate the delay the user sees, with the second biggest cost being the optimizations done, and not the parsing and validation. I'm also assuming/hoping that the bytecode would be tossed out, and the compiled output is what would be cached, making the compilation effectively a one-time cost. But if you were optimizing for web browser consumption, why not simply define web assembly as SSA, which seems to me both more in line with what I'd expect, and less effort to 'convert' to SSA? |
|
You can start to parse and compile while downloading, and some VMs might not do a full compile up front (they might just use a simple baseline for example). So download and compile times can be smaller than expected, and as a result parsing and validation can end up a significant factor in the total delay the user sees. Regarding SSA representations, they tend to have large code sizes. SSA is great for optimizing code, but not for serializing code compactly. |
|
@oridb See the comment by @titzer 'The design of WebAssembly's control flow constructs simplifies consumers by enabling fast, simple verification, easy, one pass conversion to SSA form ...' - it can generate verified SSA in one pass. Even if wasm used SSA for the encoding it would still have the burden of verifying it, of computing the dominator structure which is easy with the wasm control flow restrictions. Much of the encoding efficiency of wasm appears to come from being optimized for the common code pattern in which definitions have a single use that is used in stack order. I expect that an SSA encoding could do so too, so it could be of similar encoding efficiency. Operators such as I think wasm is not too far from being able to encode most code in SSA style. If definitions were passed up the scope tree as basic block outputs then it might be complete. Might the SSA encoding be orthogonal to the CFG matter. E.g. There could be an SSA encoding with the wasm CFG restrictions, there could be a register based VM with the CFG restrictions. A goal for wasm is to move the optimization burden out the runtime consumer. There is strong resistance to adding complexity in the runtime compiler, as it increases the attack surface. So much of the design challenge is to ask what can be done to simplify the runtime compiler without hurting performance, and much debate! |
|
Well, it's probably too late now, but I'd like to question the idea that the relooper algorithm, or variants thereof, can produce "good enough" results in all cases. They clearly can in most cases, since most source code doesn't contain irreducible control flow to start with, optimizations don't usually make things too hairy, and if they do, e.g. as part of merging duplicate blocks, they can probably be taught not to. But what about pathological cases? For example, what if you have a coroutine which a compiler has transformed to a regular function with structure like this pseudo-C: void transformed_coroutine(struct autogenerated_context_struct *ctx) {
int arg1, arg2; // function args
int var1, var2, var3, …; // all vars used by the function
switch (ctx->current_label) { // restore state
case 0:
// initial state, load function args caller supplied and proceed to start
arg1 = ctx->arg1;
arg2 = ctx->arg2;
break;
case 1:
// restore all vars which are live at label 1, then jump there
var2 = ctx->var2;
var3 = ctx->var3;
goto resume_1;
[more cases…]
}
[main body goes here...]
[somewhere deep in nested control flow:]
// originally a yield/await/etc.
ctx->var2 = var2;
ctx->var3 = var3;
ctx->current_label = 1;
return;
resume_1:
// continue on
}So you have mostly normal control flow, but with some gotos pointed at the middle of it. This is roughly how LLVM coroutines work. I don't think there's any nice way to reloop something like that, if the 'normal' control flow is complex enough. (Could be wrong.) Either you duplicate massive parts of the function, potentially needing a separate copy for every yield point, or you turn the whole thing into a giant switch, which according to @kripken is 4x slower than relooper on typical code (which itself is probably somewhat slower than not needing relooper at all). The VM could reduce the overhead of a giant switch with jump threading optimizations, but surely it's more expensive for the VM to perform those optimizations, essentially guessing how the code reduces to gotos, than to just accept explicit gotos. As @kripken says, it's also less predictable. Maybe doing that kind of transformation is a bad idea to start with, since afterward nothing dominates anything so SSA-based optimizations can't do much… maybe it's better done at the assembly level, maybe wasm should eventually get native coroutine support instead? But the compiler can perform most optimizations before doing the transformation, and it seems that at least the designers of LLVM coroutines didn't see an urgent need to delay the transformation until code generation. On the other hand, since there's a fair amount of variety in the exact semantics people want from coroutines (e.g. duplication of suspended coroutines, ability to inspect 'stack frames' for GC), when it comes to designing a portable bytecode (rather than a compiler), it's more flexible to properly support already-transformed code than to have the VM do the transformation. Anyway, coroutines are just one example. Another example I can think of is implementing a VM-within-a-VM. While a more common feature of JITs is side exits, which don't require goto, there are situations that call for side entries - again, requiring goto into the middle of loops and such. Another would be optimized interpreters: not that interpreters targeting wasm can really match those targeting native code, which at minimum can improve performance with computed gotos, and can dip into assembly for more… but part of the motivation for computed gotos is to better leverage the branch predictor by giving each case its own jump instruction, so you might be able to replicate some of the effect by having a separate switch after each opcode handler, where the cases would all just be gotos. Or at least have an if or two to check for specific instructions that commonly come after the current one. There are some special cases of that pattern that might be representable with structured control flow, but not the general case. And so on… Surely there's some way to allow arbitrary control flow without making the VM do a lot of work. Straw man idea, might be broken: you could have a scheme where jumps to child scopes are allowed, but only if the number of scopes you have to enter is less than a limit defined by the target block. The limit would default to 0 (no jumps from parent scopes), which preserves the current semantics, and a block's limit can't be greater than the parent block's limit + 1 (easy to check). And the VM would change its dominance heuristic from "X dominates Y if it is a parent of Y" to "X dominates Y if it is a parent of Y with distance greater than Y's child jump limit". (This is a conservative approximation, not guaranteed to represent the exact dominator set, but the same is true for the existing heuristic - it's possible for an inner block to dominate the bottom half of an outer one.) Since only code with irreducible control flow would need to specify a limit, it wouldn't increase code size in the common case. Edit: Interestingly, that would basically make the block structure into a representation of the dominance tree. I guess it would be much simpler to express that directly: a tree of basic blocks, where a block is allowed to jump to a sibling, ancestor, or immediate child block, but not to a further descendant. I'm not sure how that best maps onto the existing scope structure, where a "block" can consist of multiple basic blocks with sub-loops in between. |
|
FWIW: Wasm has a particular design, which is explained in just a few very significant words "except that the nesting restriction makes it impossible to branch into the middle of a loop from outside the loop". If it were just a DAG then validation could just check that branches were forward, but with loops this would allow branching into the middle of the loop from outside the loop, hence the nested block design. The CFG is only part of this design, the other being data flow, and there is a stack of values and blocks can also be organized to unwind the values stack which can very usefully communicate the live range to the consumer which saves work converting to SSA. It is possible to extend wasm to be an SSA encoding (add If this were extended to handle arbitrary CFG then might it look like the following. This is an SSA style encoding so values are constants. It seems to still fit the stack style to a large extent, just not certain of all the details. So within But would web browsers ever handle this efficient internally? Would someone with a stack machine background recognize the code pattern and be able to match it to a stack encoding? |
|
There is some interesting discussion on irreducible loops here http://bboissin.appspot.com/static/upload/bboissin-thesis-2010-09-22.pdf I did not follow it all on a quick pass, but it mentions converting irreducible loops to reducible loops by adding an entry node. For wasm it sounds like adding a defined input to loops that is specifically for dispatching within the loop, similar to the current solution but with a defined variable for this. The above mentions this is virtualized, optimized away, in processing. Perhaps something like this could be an option? If this is on the horizon, and given that producers already need to use a similar technique but using a local variable, then might it be worth considering now so that wasm produced early has potential to run faster on more advanced runtimes? This might also create an incentive for competition between the runtimes to explore this. This would not exactly be arbitrary labels and gotos but something that these might be transformed into that has some chance of being efficiently compiled in future. |
|
For the record, I am strongly with @oridb and @comex on this issue. Given the nature of WebAssembly, any mistakes you make now are likely to stick for decades to come (look at Javascript!). That's why the issue is so critical; avoid supporting gotos now for whatever reason it is (e.g. to ease optimization, which is --- quite frankly --- a specific implementation's influence over a generic thing, and honestly, I think it's lazy), and you'll end up with problems in the long run. I can already see future (or current, but in the future) WebAssembly implementations trying to special-case recognize the usual while/switch patterns to implement labels in order to handle them properly. That's a hack. WebAssembly is clean slate, so now is the time to avoid dirty hacks (or rather, the requirements for them). |
|
WebAssembly as currently specified is already shipping in browsers and toolchains, and developers have already created code which takes the form laid out in that design. We therefore cannot change the design in a breaking manner. We can, however, add to the design in a backward-compatible manner. I don't think any of those involved think At this point in time, someone with motivation needs to come up with a proposal which makes sense and implement it. I don't see such a proposal being rejected if it provides solid data.
So I'll call your bluff: I think having the motivation you show, and not coming up with a proposal and implementation as I detail above, is quite frankly lazy. I'm being cheeky of course. Consider that we've got folks banging on our doors for threads, GC, SIMD, etc—all making passionate and sensible arguments for why their feature is most important—it would be great if you could help us tackle one of these issues. There are folks doing so for the other features I mention. None for Otherwise I think |
|
Hi. I am in middle of writing a translation from webassembly to IR and back to webassembly, and I've had a discussion about this subject with people. I've been pointed out that irreducible control flow is tricky to represent in webassembly. It can prove out to be troublesome for optimizing compilers that occassionally write out irreducible control flows. This might be something like the loop under, which has multiple entry points: EBB compilers would produce the following: Next we get to translating this to webassembly. The problem is that although we have decompilers figured out ages ago, they always had the option of adding the goto into irreducible flows. Before it gets to be translated, the compiler is going to do tricks on this. But eventually you get to scan through the code and position the beginnings and endings of the structures. You end up with the following candinates after you eliminate the fall-through jumps: Next you need to build a stack out of these. Which one goes to the bottom? It is either the 'inside loop' or then it is the 'loop'. We can't do this so we have to cut the stack and copy things around: Now we can translate this to webassembly. Pardon me, I'm not yet familiar with how these loops construct out. This is not a particular problem if we think about old software. It is likely that the new software is translated to web assembly. But the problem is in how our compilers work. They have been doing the control flow with basic blocks for decades and assume everything goes. Technically the language is translated in, then translated out. We only need a mechanism that allows the values to flow across the boundaries neat without the drama. The structured flow is only useful for people intending to read the code. But for example, the following would work just as fine: The numbers would be implicit, that is.. when the compiler sees a 'label', it knows that it starts a new extended block and give it a new index number, starting to increment from 0. To produce a static stack, you could track how many items are in the stack when you encounter a jump into the label. If there ends up being inconsistent stack after a jump into the label, the program is invalid. If you find the above bad, you can also try add an explicit stack length into each label (perhaps delta from the last indexed label's stack size, if the absolute value is bad for compression), and a marker to each jump about how many values it copies in from the top of the stack during the jump. I could bet that you can't outsmart the gzip in any way by the fact how you represent the control flow, so you could choose the flow that's nice for the guys that have the hardest work here. (I can illustrate with my flexible compiler toolchain for the 'outsmarting the gzip' -thing if you like, just send me a message and lets put up a demo!) |
|
I feel like a shatterhead right now. Just re-read through the WebAssembly spec and picked up that the irreducible control flow is intentionally left out from the MVP, perhaps for the reason that emscripten had to solve the problem on the early days. The solution on how to handle the irreducible control flow in WebAssembly is explained in the paper "Emscripten: An LLVM-to-JavaScript Compiler". The relooper reorganizes the program something like this: The rational was that the structured control flow helps reading the source code dump, and I guess it is believed to help the polyfill implementations. The people compiling from webassembly will probably adapt to handle and separate the collapsed control flow. |
|
So:
|
|
It would be really nice if it were possible to jump into a loop though, wouldn't it? IIUC, if that case were accounted for then the nasty loop+br_table combo would never be needed... Edit: oh, you can make a loops without |
|
@qwertie If a given loop is not a natural loop, the wasm-targeting compiler should express it using |
Not quite: at least in SM, the IR graph is not a fully general graph; we assume certain graph invariants that follow from being generated from a structured source (JS or wasm) and often simplify and/or optimize the algorithms. Supporting a fully general CFG would either require auditing/changing many of the passes in the pipeline to not assume these invariants (either by generalizing them or pessimizing them in case of irreducibility) or node-splitting duplication up front to make the graph reducible. This is certainly doable, of course, but it's not true that this is simply a matter of wasm being an artificial bottleneck. Also, the fact that there are many options and different engines will do different things suggests that having the producer deal with irreducibility up front will produce somewhat more predictable performance in the presence of irreducible control flow. When we've discussed backwards-compatible paths for extending wasm with arbitrary goto support in the past, one big question is what's the use case here: is it "make producers simpler by not having to run a relooper-type algorithm" or is it "allow more efficient codegen for actually-irreducible control flow"? If it's just the former, then I think we probably would want some scheme of embedding arbitrary labels/gotos (that is both backwards compatible and also composes with future block-structured try/catch); it's just a matter of weighing cost/benefit and the issues mentioned above. But for the latter use case, one thing we've observed is that, while you do every now and then see a Duff's device case in the wild (which isn't actually an efficient way to unroll a loop...), often where you see irreducibility pop up where performance matters is interpreter loops. Interpreter loops also benefit from indirect threading which needs computed goto. Also, even in beefy offline compilers, interpreter loops tend to get the worst register allocation. Since interpreter loop performance can be pretty important, one question is whether what we really need is a control flow primitive that allows the engine to perform indirect threading and do decent regalloc. (This is an open question to me.) |
|
@lukewagner
For me it's the latter; my proposal expects producers to still run a relooper-type algorithm to save the backend the work of identifying dominators and natural loops, falling back to I really should gather more data on how common irreducible control flow is in practice… However, my belief is that penalizing such flow is essentially arbitrary and unnecessary. In most cases, the effect on overall program runtime should be small. However, if a hotspot happens to include irreducible control flow, there will be a severe penalty; in the future, WebAssembly optimization guides might include this as a common gotcha, and explain how to identify and avoid it. If my belief is correct, this is an entirely unnecessary form of cognitive overhead for programmers. And even when the overhead is small, WebAssembly already has enough overhead compared to native code that it should seek to avoid any extra. I'm open to persuasion that my belief is incorrect.
That sounds interesting, but I think it would be better to start with a more general-purpose primitive. After all, a primitive tailored for interpreters would still require backends to deal with irreducible control flow; if you're going to bite that bullet, may as well support the general case too. Alternately, my proposal might already serve as a decent primitive for interpreters. If you combine |
|
@comex I guess one could simply turn off whole optimization passes at the function level in the presence of irreducible control flow (although SSA generation, regalloc, and a probably a few others would be needed and thus require work), but I was assuming we wanted to actually generate quality code for functions with irreducible control flow and that involves auditing each algorithm that previously assumed a structured graph. |
|
The nested loop structure, the thing that reducibility guarantees, is
pretty much thrown away at the start. [...] I checked the current
WebAssembly implementations in JavaScriptCore, V8, and SpiderMonkey, and
they all seem to follow this pattern.
Not quite: at least in SM, the IR graph is not a fully general graph; we
assume certain graph invariants that follow from being generated from a
structured source (JS or wasm) and often simplify and/or optimize the
algorithms.
Same in V8. It is actually one of my major gripes with SSA in both
respective literature and implementations that they almost never define
what constitutes a "well-formed" CFG, but tend to implicitly assume various
undocumented constraints anyways, usually ensured by construction by the
language frontend. I bet that many/most optimisations in existing compilers
would not be able to deal with truly arbitrary CFGs.
As @lukewagner says, the main use case for irreducible control probably is
"threaded code" for optimised interpreters. Hard to say how relevant those
are for the Wasm domain, and whether its absence actually is the biggest
bottleneck.
Having discussed irreducible control flow with a number of people
researching compiler IRs, the "cleanest" solution probably would be to add
the notion of mutually recursive blocks. That would happen to fit Wasm's
control structure quite well.
|
|
Loop optimizations in LLVM will generally ignore irreducible control flow and not attempt to optimize it. The loop analysis they're based on will only recognize natural loops, so you just have to be aware that there can be CFG cycles that are not recognized as loops. Of course, other optimizations are more local in nature and work just fine with irreducible CFGs. From memory, and probably wrong, SPEC2006 has a single irreducible loop in 401.bzip2 and that's it. It's quite rare in practice. Clang will only emit a single |
|
There is no single-pass verification algorithm for irreducible control flow
that I am aware of. The design choice for reducible control flow only was
highly influenced by this requirement.
As mentioned earlier, irreducible control flow can be modeled at least two
different ways. A loop with a switch statement can actually be optimized
into the original irreducible graph by a simple local jump-threading
optimization (e.g. by folding the pattern where an assignment of a constant
to a local variable occurs, then a branch to a conditional branch that
immediately switches on that local variable).
So the irreducible control constructs are not necessary at all, and it is
only a matter of a single compiler backend transformation to recover the
original irreducible graph and optimize it (for engines whose compilers
support irreducible control flow--which none of the 4 browsers do, to the
best of my knowledge).
Best,
-Ben
…On Thu, Apr 20, 2017 at 5:20 AM, Jakob Stoklund Olesen < ***@***.***> wrote:
Loop optimizations in LLVM will generally ignore irreducible control flow
and not attempt to optimize it. The loop analysis they're based on will
only recognize natural loops, so you just have to be aware that there can
be CFG cycles that are not recognized as loops. Of course, other
optimizations are more local in nature and work just fine with irreducible
CFGs.
From memory, and probably wrong, SPEC2006 has a single irreducible loop in
401.bzip2 and that's it. It's quite rare in practice.
Clang will only emit a single indirectbr instruction in functions using
computed goto. This has the effect of turning threaded interpreters into
natural loops with the indirectbr block as a loop header. After leaving
LLVM IR, the single indirectbr is tail-duplicated in the code generator
to reconstruct the original tangle.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#796 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ALnq1K99AR5YaQuNOIFIckLLSIZbmbd0ks5rxkJQgaJpZM4J3ofA>
.
|
|
I can also say further that if irreducible constructs were to be added to
WebAssembly, they would not work in TurboFan (V8's optimizing JIT), so such
functions would either end up being interpreted (extremely slow) or being
compiled by a baseline compiler (somewhat slower), since we will likely not
invest effort in upgrading TurboFan to support irreducible control flow.
That means functions with irreducible control flow in WebAssembly would
probably end up with much worse performance.
Of course, another option would for the WebAssembly engine in V8 to run the
relooper to feed TurboFan reducible graphs, but that would make compilation
(and startup worse). Relooping should remain an offline procedure in my
opinion, otherwise we are ending with up inescapable engine costs.
Best,
-Ben
…On Mon, May 1, 2017 at 12:48 PM, Ben L. Titzer ***@***.***> wrote:
There is no single-pass verification algorithm for irreducible control
flow that I am aware of. The design choice for reducible control flow only
was highly influenced by this requirement.
As mentioned earlier, irreducible control flow can be modeled at least two
different ways. A loop with a switch statement can actually be optimized
into the original irreducible graph by a simple local jump-threading
optimization (e.g. by folding the pattern where an assignment of a constant
to a local variable occurs, then a branch to a conditional branch that
immediately switches on that local variable).
So the irreducible control constructs are not necessary at all, and it is
only a matter of a single compiler backend transformation to recover the
original irreducible graph and optimize it (for engines whose compilers
support irreducible control flow--which none of the 4 browsers do, to the
best of my knowledge).
Best,
-Ben
On Thu, Apr 20, 2017 at 5:20 AM, Jakob Stoklund Olesen <
***@***.***> wrote:
> Loop optimizations in LLVM will generally ignore irreducible control flow
> and not attempt to optimize it. The loop analysis they're based on will
> only recognize natural loops, so you just have to be aware that there can
> be CFG cycles that are not recognized as loops. Of course, other
> optimizations are more local in nature and work just fine with irreducible
> CFGs.
>
> From memory, and probably wrong, SPEC2006 has a single irreducible loop
> in 401.bzip2 and that's it. It's quite rare in practice.
>
> Clang will only emit a single indirectbr instruction in functions using
> computed goto. This has the effect of turning threaded interpreters into
> natural loops with the indirectbr block as a loop header. After leaving
> LLVM IR, the single indirectbr is tail-duplicated in the code generator
> to reconstruct the original tangle.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#796 (comment)>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ALnq1K99AR5YaQuNOIFIckLLSIZbmbd0ks5rxkJQgaJpZM4J3ofA>
> .
>
|
|
@conrad-watt as you just mentioned in the CG meeting, I think we'd be very interested in seeing details on what your multi-loop would look like. |
|
@aardappel yeah, life has come at me fast, but I should do this in the next meeting. Just to emphasise that the idea isn't mine since @rossberg originally sketched it in response to the first draft of funclets. |
|
One reference that might be instructive is a kind of a bit dated, but generalizes familiar notions of loops to handle irreducible ones using DJ graphs. |
I'm carving out a fairly large time slot for discussion, but if too many other items come along this should probably be reduced. Conversely, I'm sure the [debate](WebAssembly/design#796) has the potential to take up all available time, so this should probably be timetabled last. To be clear, not planning to poll anything.
I'm carving out a fairly large time slot for discussion, but if too many other items come along this should probably be reduced. Conversely, I'm sure the [debate](WebAssembly/design#796) has the potential to take up all available time, so this should probably be timetabled last. To be clear, not planning to poll anything.
|
We've had a couple of discussion sessions about this in the CG, and I've written up a summary and follow-up document. Because of the length I've made it a separate gist. https://gist.github.com/conrad-watt/6a620cb8b7d8f0191296e3eb24dffdef I think the two immediate actionable questions (see the follow-up section for more details) are:
There is probably also a more free-wheeling discussion to be had on the consequences of the exception handling issues I discuss in the follow-up document, and of course standard bikeshedding about semantic details if we move forward with anything concrete. Because these discussions may branch somewhat, it may be appropriate to spin some of them into issues in the funclets repository. |
|
I am very happy to see progress on this issue. A huge "Thank you" to all people involved!
I'd like to caution a bit against circular reasoning: Programs that currently have bad performance are less likely to occur "in the wild" for exactly this reason. I think most Go programs should benefit a lot. The Go compiler either needs WebAssembly coroutines or |
|
Precompiled regular-expression matchers, along with other precompiled state machines, often result in irreducible control flow. It's hard to say whether or not the "fusion" algorithm for Interface Types will result in irreducible control flow. |
|
I agree that testing on modified toolchains + VMs would be optimal. But we can compare current wasm builds to native builds which do have optimal control flow. Not So Fast and others have looked at this in various ways (performance counters, direct investigation) and have not found irreducible control flow to be a significant factor. |
|
More specifically, they didn't find it to be a significant factor for C/C++. That might have more to do with C/C++ than with the performance of irreducible control flow. (I honestly don't know.) It sounds like @neelance has reason to believe the same would not be true for Go. My sense is that there are multiple facets to this problem, and its worthwhile tackling it through multiple directions. First, it sounds like there's a general issue with the generatability of WebAssembly. Much of that is caused by WebAssembly's constraint to have a compact binary with efficient type-checking and streaming compilation. We could address this issue at least partly by developing a standardized "pre"-WebAssembly that is easier to generate but which is guaranteed to be translatable to "true" WebAssembly, ideally through just code duplication and insertion of "erasable" instructions/annotations, with at least some tool providing such translation. Second, we can consider what features of "pre"-WebAssembly are worth directly incorporating into "true" WebAssembly. We can do this in an informed manner because we will have "pre"-WebAssembly modules that we can analyze before they have been contorted into "true" WebAssembly modules. |
|
Some years ago I tried compiling a particular bytecode emulator for a dynamic language (https://github.com/ciao-lang/ciao) to webassembly and the performance was far from optimal (sometimes 10 times slower than the native version). The main execution loop contained a large bytecode dispatch switch, and the engine was finely tuned for decades to run on actual hardware, and we make heavy use of labels and gotos. I wonder if this kind of software would benefit from support for irreducible control flow or if the problem was another one. I didn't have time to do further investigation but I'd happy to try again if things are known to have improved. Of course I understand that compiling other languages VM to wasm is not the main use case, but I'd be good to know if this will be eventually feasible, specially since universal binaries that run efficiently, everywhere, is one of the promised advantages of wasm. (Thanks and apologies if this particular topic has been discussed in some other issue) |
|
@jfmc My understanding is that, if the program is realistic (i.e. not contrived in order to be pathological) and you care about its performance, then it is a perfectly valid use case. WebAssembly aims to be a good general-purpose target. So I think it would be great to gain an understanding of why you saw such significant slowdown. If that happens to be due to restrictions on control flow, then that would be very useful to know in this discussion. If it happens to be due to something else, then that would still be useful to know for how to improve WebAssembly in general. |
|
TinyCC with WebAssembly backend would be awesome... a fast in-browser C compiler https://lists.gnu.org/archive/html/tinycc-devel/2020-02/msg00017.html Another motivating reminder of what could be, if & hopefully when. |
|
Python port to WebAssembly would benefit greatly from computed gotos. When we added computed goto support to CPython's ceval loop (core bytecode VM loop) many years ago, we saw an overall performance improvement of approx. 15 to 20% on platforms that use GCC and clang. I assume that addition of computed goto support in the WASM compiler tool chain would result in similar speedups. Eli Bendersky's old blog post explains how Python's ceval loop benefits from more efficient machine code from computed gotos. |
|
I read the blog post, and it seems to be a result of their version of C not having an "unreachable" statement. However, in wasm you won't see those improvements since even a computed goto has to have validity checks unless the engine can infer them statically. |
|
@taralx I think the more important part is the branch prediction:
Also:
This is the point of the funclet proposal, which would be perfect for our use case. The entire table is statically determined so all of the validity checks are static. |
|
I think I may have a solution, although it does involve changing the way WebAssembly works. Instead of compiling to IR that might be JIT'd, executed, or what have you, change the WASM standard to be a proper assembly, ie with jumps and all that. This makes it hard to recompile to a faster assembly, but some people wouldn't need to recompile if it already had jumps. For those that want to use it as an IR, there should be a way to make a pointer manifest to be included with the assembly somehow- to aid in recompiling to client code. This pointer manifest would do the same job as Amendment A Reading through, I have seen comments that this is not possible given the fact that control flow is stable. This is a niche that many would like to be filled, however. |
|
If making a language (like an esolang based on early BASICs), goto is definitely almost a necessity for the backend. It's currently looking like the project i'm working on will need to sacrifice some accuracy to avoid deeply nesting everything in blocks. This format also seems to work better for multi-backend compilers that might already compile everything into a load of jump instructions emitted as assembly, GNU Lightning IR, or some other unstructured format. |
To clarify, since I think there is a persistent misunderstanding here: arbitrary jumps would not make recompiling to faster assembly more difficult, since it is reasonably simple to detect irreducible control flow and compiling reducible control flow constructs expressed with simple jumps is arguably easier than with higher-level, nested control flow. That's why every compiler apart from V8 converts the source language's high-level control flow to jumps before optimisation (and well before translating to assembly). Irreducible control flow could make some regalloc constraint solving a bit more of a hassle but worst-case it would only make irreducible control flow a little bit slower than reducible, instead of the current situation where irreducible control flow is entirely inexpressible and therefore it must be emulated by compilers targeting Wasm. Let me reiterate: arbitrary jumps which happen to express reducible control flow are just as easy to compile as high-level control flow constructs. V8 is the only compiler to my knowledge which would find it particularly difficult to compile irreducible control flow, and the relooper algorithm which LLVM uses when generating Wasm could just be integrated into V8 itself if gotos were implemented into the Wasm standard. Not to mention that it's years-old knowledge that V8 can't handle irreducible control flow - they might have updated the internals in the past 3+ years, in which case there is no compiler to my knowledge that would find it harder to handle gotos than nested control flow. |
In theory yes, but as mentioned 6 years ago in this discussion, that would move a significant amount of work from the toolchain side to the client. In general wasm tries to do the opposite (in order to achieve fast startup, and to reduce the risk of client-side bugs). Meanwhile, in the years since the earlier discussions here, wasm has added more dependencies on structured control flow, like non-nullable locals (quick overview) and exceptions. All those things would either not be compatible with arbitrary gotos or require work to figure out how to support them there. Overall it's not impossible to support irreducibility but it is getting harder over time I think. |
|
I used to think this issue was necessary. Then I had the idea of an inverse relooper algorithm. Every compiler would run relooper, and V8 can run that output directly and call it a day. Runtimes that support irreducible CFGs can detect that relooper was used from that giant loop and switch, and they can invert the algorithm to get back the original CFG |
|
@danielzgtg This is what I was suggesting in the comment linked above. The algorithm to undo the loop over the switch is a relatively straightforward generalization of tail duplication, a well-known compiler optimization that can be done by the engine after it's created a CFG. If that CFG is in SSA form, it will look like this: Block Then we end up with: Which is your classic irreducible loop. All this is to say, it's always been within engines' power to do this optimization and reintroduce irreducible loops, if they are prepared to handle them throughout the rest of their backends. For the record, AFAICT, all the web engines that I am aware of have exactly the same limitations as TurboFan in their optimizing tiers. None support irreducible control flow--but could, given the above. |
and others
I'd like to point out that I haven't been involved in the web assembly effort,
and I'm not maintaining any large or widely used compilers (just my own
toy-ish language, minor contributions to the QBE compiler backend, and an
internship on IBM's compiler team), but I ended up getting a bit ranty, and
was encouraged to share more widely.
So, while I'm a bit uncomfortable jumping in and suggesting major changes
to a project I haven't been working on... here goes:
My Complaints:
When I'm writing a compiler, the first thing that I'd do to with the high level
structure -- loops, if statements, and so on -- is validate them for semantics,
do type checking and so on. The second thing I do with them is just throw them
out, and flatten to basic blocks, and possibly to SSA form. In some other parts
of the compiler world, a popular format is continuation passing style. I'm not
an expert on compiling with continuation passing style, but it neither seems to
be a good fit for the loops and scoped blocks that web assembly seems to have
embraced.
I'd like to argue that a flatter, goto based format would be far more useful as
a target for compiler developers, and would not significantly hinder the
writing of a usable polyfill.
Personally, also I'm not a big fan of nested complex expressions. They're a bit
clunkier to consume, especially if inner nodes can have side effects, but I
don't strongly object to them as a compiler implementer -- The web assembly
JIT can consume them, I can ignore them and generate the instructions that map
to my IR. They don't make me want to flip tables.
The bigger problem comes down to loops, blocks, and other syntactic elements
that, as an optimizing compiler writer, you try very hard to represent as a
graph with branches representing edges; The explicit control flow constructs
are a hindrance. Reconstructing them from the graph once you've actually done
the optimizations you want is certainly possible, but it's quite a bit of
complexity to work around a more complex format. And that annoys me: Both the
producer and the consumer are working around entirely invented problems
which would be avoided by simply dropping complex control flow constructs
from web assembly.
In addition, the insistence of higher level constructs leads to some
pathological cases. For example, Duff's Device ends up with horrible web
assembly output, as seen by messing around in The Wasm Explorer.
However, the inverse is not true: Everything that can be expressed
in web assembler can be trivially converted to an equivalent in some
unstructured, goto based format.
So, at the very least, I'd like to suggest that the web assembly team add
support for arbitrary labels and gotos. If they choose to keep the higher
level constructs, it would be a bit of wasteful complexity, but at least
compiler writers like me wold be able to ignore them and generate output
directly.
Polyfilling:
One of the concerns I have heard when discussing this is that the loop
and block based structure allows for easier polyfilling of web assembly.
While this isn't entirely false, I think that a simple polyfill solution
for labels and gotos is possible. Whiie it might not be quite as optimal,
I think that it's worth a little bit of ugliness in the bytecode in order
to avoid starting a new tool with built in technical debt.
If we assume an LLVM (or QBE) like syntax for web assmembly, then some code
that looks like:
might compile to:
This could be polyfilled to Javascript that looks like:
Is it ugly? Yeah. Does it matter? Hopefuly, if web assembly takes off,
not for long.
And if not:
Well, if I ever got around to targeting web assembly, I guess I'd generate code
using the approach I mentioned in the polyfill, and do my best to ignore all of
the high level constructs, hoping that the compilers would be smart enough to
catch on to this pattern.
But it would be nice if we didn't need to have both sides of the code generation
work around the format specified.
The text was updated successfully, but these errors were encountered: