v2.2.0

Blazing Bulldog

What's new?

• Full comptime support including inline ASM now possible with our native SSA-based x86_64 backend.
• Native REPL written in Ideka
• 31% compiler binary size reduction

Native x86_64 backend (LLVM replacement)

Ideka's compiler generates its own x86_64 machine code through an SSA pipeline (issa.idk / ix64.idk, plus iasm/ilink for assembling and linking) instead of emitting LLVM IR.

Optimization passes added/tuned on this backend:

• Rolling copy loops
• Strength reduction (s_sr)
• BMI instruction selection (andn, bzhi, rorx, blsr, blsi via flagless VEX forms)
• Redundant-load removal (DOMLD): implemented, measured a negligible gain at a real compile-time cost, left disabled
• Loop-invariant code motion (LICM): implemented, measured a regression (+0.27% dynamic instruction count), reverted/left disabled
• Constant rematerialization / index sign-extension hoisting
• An inline keyword path: the compiler already honors inline on a function signature; functions needing it were simply marked, no change to force inlining was made

Measured generated-code gains:

• t4 test program dynamic instruction count (Ir, via valgrind callgrind): EQCHAIN pass, -2.35%; marking hot small functions inline, a further -0.13% (479,879,364 to 479,255,432).
• Compiler binary size (on my machine): 2.6 MB to 1.8 MB, about 31% smaller, from this optimization work.

• Block merging (SSA_OPT_MERGE): a block that ends in a jump to a block nothing else reaches absorbs it, removing the jump and letting a compare fuse with the branch/select that uses it (a cmp, setl, movzbl, testb, cmove sequence at function entry collapses to cmp + cmov). Compiler assembly about 1% smaller.
• Loop test repetition (SSA_OPT_ROTATE): a jump back to a loop test that is only a compare and a conditional jump now repeats the test in place, so a loop needs one jump per iteration instead of two. Self-compile time: 21.5s to 19.7s, about 8% faster (most of that 8% from this pass).
• Phi cleanup bug fix: a constant-folded branch could leave a stale phi entry for the removed path, causing a miscompile (a dropped switch optimization and double-extended values) in the compiler built by that compiler, undetected by the test suite, only caught by the fixed-point check. Fixed with s_prunephi, run whenever the CFG is rebuilt.
• Branch-on-negation: if (!f()) now tests the value directly and inverts the jump, removing a movzbl/xor/test per such branch (the compiler's own xorl $1 count dropped from 1,663 to 1,085). A first attempt that swapped branch targets instead of inverting the branch flag made the compiler about 6% slower, by putting the cold path first in block order; switched to an inversion flag that leaves layout alone.
• Block layout pass: reorders blocks after register allocation so each is followed by its most likely successor (same/deeper-loop successor before a loop exit; ud2-ending blocks last; everything else keeps relative order). Self-compile: 22.7s to about 21.1s, about 6% faster; unconditional jumps dropped 8% (15,116 to 13,908).
• Byte-switch range-check removal: a switch on a byte already covers all 256 values via its jump table, so the cmpq $255,%r11; ja default guard before the table jump was redundant and is now removed (applies to every jump-table switch in the compiler, including the lexer's per-character switch, the hot one). Self-compile: 21.7s to 20.8s, about 3-4% faster.
• Indexing (GEP lowering): shift instead of multiply for power-of-two element sizes other than 1/2/4/8 (compiler's own imulq count 417 to 352), and one sign-extend reused per block for repeated same-index accesses (movslq count 7,318 to 6,554). No measurable self-compile or kernel-benchmark change; smaller, cleaner generated code only. Full value-range analysis to drop the sign-extend entirely was identified as a larger, riskier follow-up with a likely small payoff and was not done.
• Constant stores and nested address calculations: a store of a constant to a folded address now emits a direct immediate store instead of building the constant in a register first (about 4,600 sites in the compiler); table[i].field now folds to a single addressing mode instead of a separate lea plus offset load. Compiling hashmap_test: 1.112 billion to 1.082 billion dynamic instructions, about 2.7% fewer; generated assembly about 3.4% smaller.
• Fill-loop rewrite: a loop of the shape i=0; while(i<n){p[i]=c; i++;} (0-based, unsigned 64-bit or signed 32/64-bit count, body only the store and increment, index unused after) compiles to a single rep stosb.
• Neighboring one-byte constant stores merged: adjacent single-byte constant stores to one stack slot now become 4-, 2-, and 1-byte immediate stores (also merges through a pointer register, for runs of at least two bytes); a 43-character string literal went from 43 movb to 11 movl instructions. Compiler's own output: 500k to 487k lines; compiler binary about 4% smaller.
• Bootstrap pass 3 speedup: pass 3 of the bootstrap (the compiler compiling itself) went from 6-7.5s to about 4s; pass 4 unchanged at roughly 1.5-2.5s. Two fixes:
  • ptr_overload_call was the largest cost: it re-scanned every function in the module and re-parsed each function name for every call site with arguments. Replaced with ov_index_sync, which indexes each function once (by overload key and its __-suffixes), so the candidate list for the second loop comes from a per-key index instead of a rescan; first function in module order still wins, result identical.
  • ptr_ovl_zero did the same kind of linear scan; indexed the same way, same dotted-keys-first priority.
  • table_grow in allocators.idk was zeroing freshly-grown table pages that the OS already hands back zeroed; that redundant zero loop was removed (stays inside the existing O(1) design of the allocator).

Benchmarks?

Allocator (fmalloc/ffree) vs C

The custom allocator (fmalloc/ffree, stdheap: size-class bins, 32-byte block-table entries, bump slabs doubling 4 MB to 64 MB, hash table with backward-shift removal) was benchmarked against glibc malloc/free, using the real tests/heap_stress.idk test compiled and run through the actual bootstrapped Ideka compiler, and a C port of the same allocator design (/tmp/cb/fmalloc.c, /tmp/cb/heap_stress.c) running the identical workload (same PCG32 generator, same alloc/free patterns, same verification sums) compiled with gcc and clang.

Same-machine comparison (this cloud container)

Ideka, gcc, and clang were all built and run in the same container:
CPU reported only as "Intel(R) Xeon(R) Processor" (2 vCPUs, no hyperthreading, ~5400 BogoMIPS), 7.8 GiB RAM; exact CPU model and memory type/speed are not exposed by this VM. Three runs each, after the SSA backend optimization work above.

Test	Ideka (fmalloc)	gcc -O2	clang -O2	gcc -O3 -march=native	clang -O3 -march=native
1M x 64-byte pairs	5.8-5.9 ms / 10.6k-10.7k MB/s	6.4-7.0 ms / 8.7k-9.5k MB/s	6.7-6.8 ms / 9.0k-9.2k MB/s	6.5-6.7 ms / 9.1k-9.4k MB/s	6.5-6.9 ms / 8.8k-9.4k MB/s
1M mixed pairs	6.1-8.1 ms / 248k-330k MB/s	18.5-21.9 ms / 89.3k-105k MB/s	18.4-18.5 ms / 105k-106k MB/s	18.2-18.8 ms / 104k-107k MB/s	18.5-19.3 ms / 101k-105k MB/s
Fast-path reuse	5.8-6.5 ms / 9.5k-10.7k MB/s	6.4-6.6 ms / 9.3k-9.5k MB/s	6.6-6.9 ms / 8.8k-9.5k MB/s	6.4-6.6 ms / 9.2k-9.5k MB/s	6.6-6.7 ms / 9.1k-9.3k MB/s
64-byte write/read-back	19.2k-20.0k MB/s	11.0k-11.3k MB/s	15.3k-15.7k MB/s	15.7k-16.3k MB/s	15.6k-15.9k MB/s
Mixed write/read-back	35.1k-35.6k MB/s	25.2k-27.0k MB/s	30.6k-33.1k MB/s	27.4k-27.5k MB/s	34.0k-34.1k MB/s
Stream write	6.8k-7.6k MB/s	6.7k-6.8k MB/s	6.8k-6.9k MB/s	7.0k-7.7k MB/s	7.3k-7.8k MB/s
Stream read	14.6k-17.1k MB/s	14.4k-15.2k MB/s	14.8k-15.8k MB/s	16.4k-17.1k MB/s	16.3k-18.7k MB/s

On this machine, Ideka leads gcc and clang at every optimization level on the timed alloc/free tests and on 64-byte/mixed read-back bandwidth; it is roughly level with the C builds on stream write/read bandwidth.
-O3 -march=native did not uniformly help gcc/clang over their own -O2 numbers here (mixed-pairs alloc/free got faster; 64-byte write/read-back bandwidth barely changed for clang and rose for gcc).

Ideka REPL (irepl.idk)

The Python REPL (irepl.py) was ported to Ideka itself (irepl.idk): Term, WinCon, Editor, ReplSession, and the repl_* functions (coordinate handling, value representation, emission printing, command handling, parsing, running a block, directive handling, main loop). It compiles and imports through the same pipeline the compiler itself uses for comptime (IDKVMCodegen.compile_comptime/compile_import, register_functions, vm.import_unit, vm.execute), reusing ivmcodegen.idk, iparser.idk, and ipreprocess.idk directly rather than reimplementing them.

Fixed while porting and testing against Windows output:

• Missing imports and keyword collisions (from/has are reserved), missing __init/__expr/__exit on ported objects, stdout buffering so output showed up live.
• #import/#def preprocessor directives now work when typed at the REPL prompt, split out of the input before it is wrapped in a comptime block, so arbitrary Ideka source can be thrown at the REPL, not just expressions.
• Windows-specific: missing console.idk import on that platform, missing Term.at_line_start, leftover text on redraw, and a cursor-reset-to-top bug traced to an i16 left-shift bug in the console coordinate packing (make_coord), fixed with a dedicated repl_coord helper.
  using namespaces and locals now persist across REPL submissions, instead of resetting each time.