[css-typed-om] Ensure serialization of deeply nested math operators d…

[achristensen07]

…oes not crash

This comes from a WebKit fix in https://bugs.webkit.org/show_bug.cgi?id=239562

[tabatkins]

I'd be fine with approving this, but in general we do allow for OOMs and such to just happen, so long as they're for reasonably excessive cases. I think a 100k-deep stack of OM classes is such a case; you're looking at a serialization around 1MB in size. ^_^

Correctly serializing arbitrarily-deep trees is cool, but I do wonder if a smaller limit might capture reasonable use-cases sufficiently well while still allowing more limited impls to work. If an impl relies on recursion, I guess handling a stack of 1k depth might be about the limit one can reasonably expect? Python's stack depth is 1k, at least.

[achristensen07]

I didn't verify the output of the serialization, so an implementation could make a maximum stack depth or use a non-recursive algorithm and still pass.