AOT & IR Cache Planning

[skmp]

This is a high level ticket, to track the work that needs to be done for a fairly complete aot/ir cache setup. Follow up from #47 #693

Current state

• FEXLoader can capture IR to aot files via --aotircapture
• FEXLoader can load IR from aot files via --aotirload
• FEXLoader can pre-process an entire elf with --aotirgenerate
• IR loading is done per executable file (.so or otherwise)
• IR loading depends on mmap hooks to detect when binaries are loaded
• IR is loaded via mmap w/ index
• Used modules create a .path entry in ~/.fex-emu/aotir/
• Scripts/FEXUpdateAOTIRCache.sh reads .path files and generates .aotir files for the matching elfs/so files

Multi threaded AOTIR generation

• There's a POC branch, needs multiple thread contexts and some other tweaks

Streamable AOTIR generation

• Move the index to the end of the file
• Stream writes

Precompiled binary caching (~ 2-4 weeks to reviewable code)

• Needs our jit to be relocation-aware
• Needs our codegen to be relocation-optimized
• Needs similar logic to mmap-based ir loading for the metadata
• Needs relocation information to be stored and parsed and applied
  • Preferably on a per-block use basis, to avoid stutters in large files
• Should introduce a FEXAOTCompiler to compile IR caches to binary caches
• Should introduce a new cache loading mode, --aotbin-load or such that loads binary caches

[skmp]

Did a couple of days of research on the binary caching. There are some further complications.

If binary cache is per process, then there's several conflicts around.

• Per thread SMC detection, invalidation (hard, blocker)
• Per thread Block Linking (blocker)
• Per thread Dispatcher (involved but doable)
• A few per thread constants (easy to fix)

[skmp]

#976 and #996 Implement the first milestone of, and make AOT IR usable for large programs.