Performance benchmarks and comparisons to other compiler projects

[matu3ba]

The text on performance is nice and all, but doesnt show benches + time differences for the intended use case of sandboxing + jitting.

• Bytecode-interpreter: Comparisons, ie against https://wavm.github.io/ would be nice (It also claims safety, but I am unsure about binary sizes). Not sure which benchmarks are best for this kind of comparison.
• Optimizing compiler: How does it currently perform vs luajit? You could port the microbenchmarks suite from julia.

For many explanation like "Reduced garbage collector" its not evident to me with examples how "competitors" do it. It might make sense to create references or elaborate this as examples.

[zeux]

I don't think WAVM is an interesting target for us. We don't implement a JIT right now, the language is completely different, etc.

Benchmark results also depend on a compiler, for us clang vs MSVC vs gcc can result in different performance, something like LuaJIT is a more stable target because all time-critical portions there are implemented in assembly. Posting benchmark comparisons with different languages is hard! Translating languages precisely can be difficult.

In general performance documentation is provided as an illustration of techniques we had to use to improve performance. Luau doesn't claim to be superior to any given implementation or language, short of vanilla Lua where we can confidently say we're much faster than any other vanilla version. LuaJIT, especially in JIT mode, is faster than Luau at this point.

As for garbage collectors, it's difficult to explain this as well! Many garbage collectors in other languages aren't incremental, but some are generational or concurrent. At some point we will probably write a separate document on garbage collection, but we're too early in the process to describe all tradeoffs well - we have current and planned work to improve garbage collection throughput.

[zeux]

I'm going to close this for now, since I don't think there's any specific actionable items for us to take today, and we already have plans to keep expanding our performance documentation as more changes land or solidify. Please feel free to ask specific questions in discussion forum!

[dumblob]

I'd slightly adjust the goal of this issue and ask for showing non-rigorous benchmarks simply by adding Luau to https://github.com/kostya/benchmarks (which is an attempt to side-step the benchmarking issues one would need to solve in rigorous benchmarks; it's also developed & maintained for years already unlike most other benchmarks).

@zeux does this sound actionable?