Skip to content

OLLVM-Next (Ensia)

License: AGPL v3 DOI Discord Server Scc Count Badge Code

⚠️ ETHICAL USE WARNING: This is a high-strength industrial compiler obfuscation tool. Please read our Ethics & Disclaimer Notice before use.

OLLVM-Next (Ensia) is a modern, high-resilience LLVM-based compiler obfuscation framework. Continuing the lineage of the Hikari, Hikari-LLVM15, and Hikari-LLVM19 projects, Ensia completely redesigns the core transformation engine for modern LLVM toolchains (LLVM 21+, including LLVM 21, 22, and 23+).

Ensia supports cross-platform code protection across Linux, Windows (MSVC, clang-cl, MinGW), macOS, iOS, and Android, with first-class architecture support for x86_64, AArch64 (ARM64), and i386.


Core Philosophy: Eliminating Single Points of Failure (SPOF)

Traditional VM-based obfuscators (Virtualizers) encapsulate target logic within a custom interpreter loop. While difficult to inspect by hand, they introduce a catastrophic Single Point of Failure (SPOF): once an analyst devirtualizes the bytecode opcode dispatch table or extracts the central interpreter handler loop, all protected routines collapse at once.

Ensia completely rejects single-point interpreter designs. Instead, it enforces Non-Linear SMT / Symbolic Solver State-Space Explosion and Dynamic Taint Analysis (DTA) Neutralization through an interleaved composition of distributed passes:

  1. Non-Linear Cascading Pipeline: Passes are scheduled in a strict 12-stage non-linear sequence where each pass transforms and multiplies the entropy generated by previous passes.
  2. Branchless Algebraic State Transitions: Control Flow Flattening (CFF) and Chaos State Machines (CSM) eliminate volatile memory markers and naked select branches, binding state updates directly to condition bitmasks and stateful contextual barriers.
  3. Stateful Contextual Barriers: Static barrier signatures are completely eliminated. Barriers are dynamically derived from runtime state (stack canary %fs:0x28 / %gs:0x14, stack alignment masks %rsp & 15, Thread Control Block self-pointers, and ARM64 tpidr_el0), backed by isolated per-barrier stack allocations to guarantee zero spill corruption.
  4. 3-Tier Anti-Taint & Data-Flow Entanglement: Neutralizes dynamic taint analysis engines (Triton, BAP, angr) via indirect Identity LUTs, implicit control-flow bit laundering, and SIMD vector lane diffusion.
  5. Memory-Dump Resistance (Anti-Dump): Plaintext buffers from String Encryption are automatically zeroized at function exits (ReturnInst / ResumeInst) using isVolatile memory scrubbing and atomic status locks, with decryption stubs tagged synthetic to eliminate MBA blowup.
  6. Triple-Scheme Constant Encryption: Combines Bivariate MBA, 4-round Feistel non-linear networks, and dynamic anti-debug token (adb.tok) entanglement without combinatorial code bloat.

Comprehensive Pass Suite (15 Passes)

Pass ID CLI Flag Env Var Description
ANTIHOOK -enable-antihook ANTIHOOK=1 Dual-defense integrity architecture: Entry Prologue Guard (0xE9, 0xEB, 0xCC, 0x68, 0xFF 0x25, 0x48 0xB8) + Scattered In-Flight CFG Auditing, embedded code self-check with data-flow entanglement (T_env / T_exp), direct syscall bypass, and 3-tier anti-taint engine.
ACDOBF -enable-acdobf ACDOBF=1 Objective-C & Swift metadata scrambling, Fisher-Yates method list shuffling, randomized selector hashing, and dummy selector injection.
FCO -enable-fco FCO=1 Function Call Obfuscation: replaces direct calls with runtime dlopen/dlsym (POSIX) or GetProcAddress (Windows), completely eliminating external symbol imports from binary headers.
ADB -enable-adb ADB=1 Zero-dependency debugger detection: /proc/self/status TracerPid, PTRACE_TRACEME, hardware debug registers (DR0-DR7), single-step trap flag (EFLAGS.TF with red-zone stack preservation), and immediate violent termination (SYS_exit_group(137) / inline hardware traps).
STRCRY -enable-strcry STRCRY=1 Dual-layer Vernam OTP + Rijndael GF(2^8) Galois Field cipher with unordered dynamic inlined decryption stubs and Anti-Dump memory zeroization at function return/resume.
CONSTENC -enable-constenc CONSTENC=1 Two-phase constant encryption: Scheme A (Bivariate MBA k-share), Scheme B (4-round Feistel non-linear mixing), and Scheme C (Dynamic AntiDebug token %adb.tok entanglement with DominatorTree validation).
SUBOBF -enable-subobf SUBOBF=1 Instruction Substitution: replaces basic arithmetic and bitwise operations with complex algebraic and rotate-identity trees.
MBAOBF -enable-mbaobf MBAOBF=1 Mixed Boolean-Arithmetic: transforms expressions into multivariate non-linear boolean-arithmetic over ring Z/(2^n)Z with 42 built-in identities, Point-to-Point (BPP) dataflow tracking, context-dependent non-zero polynomial noise ($P(x, y) \cdot (\text{ctx} \oplus K)$), and stateful contextual barriers.
SPLITOBF -enable-splitobf SPLITOBF=1 Basic Block Splitting: slices blocks across instruction chains, enforced by mandatory opaque predicate chaining (((seed * (seed + 1)) & 1) == 0) with bogus loops to prevent simplifycfg collapse, and injects inline-ASM stack pointer confusion.
BCFOBF -enable-bcfobf BCFOBF=1 Bogus Control Flow: injects non-patchable hardware opaque predicates (CPUID SSE bit, RDTSC parity, CNTPCT_EL0, polymorphic barriers), dead-code loops, and entropy chains.
CSMOBF -enable-csmobf CSMOBF=1 Chaos State Machine: transforms CFGs into a chaotic dynamical system governed by the quadratic logistic map in full Q32 fixed-point arithmetic ($2^{32}$ state space) with multi-step cellular automata attractor basins (diffuseState), modular inverse decoding (modInverse32), dynamic data-flow feedback (DFB), and optional 2-level nested dispatch.
CFFOBF -enable-cffobf CFFOBF=1 Control Flow Flattening: fallback flattening with Branchless Algebraic State Transitions (mask = 0 - zext(cond), diff = true ^ false, next = false ^ (mask & diff)), eliminating volatile single points of failure.
VOBF -enable-vobf VOBF=1 Vector Obfuscation: lifts scalar arithmetic and comparisons into 128/256/512-bit SIMD vector space with pseudo-random lane noise, shufflevector bijective permutations, and Vector Taint Diffusion.
INDIBRAN -enable-indibran INDIBRAN=1 Indirect Branching: encrypts basic block jump targets via Knuth multiplicative golden-ratio hashing, runtime Newton-Raphson modular inverse decode, and encrypted jump table arrays.
FUNCWRA -enable-funcwra FUNCWRA=1 Function Wrapper: wraps function entry points with polymorphic proxy trampolines (IdentityNoise, ArgShuffle, RetMask), XOR-scrambled function pointers, and enforced noinline optnone.

The 12-Stage Non-Linear Cascading Pipeline

Ensia schedules passes to maximize cascading complexity. Each pass treats the obfuscated output of previous passes as input, creating an exponential barrier to symbolic deobfuscation:

 1. AntiHooking & AntiClassDump     -> Dual-defense AntiHook (Entry Prologue Guard + Scattered In-Flight Auditing + Embedded Integrity Self-Check with Data-Flow Entanglement), direct syscalls, dynamic execution tokens (T_env), ObjC metadata scrambling
 2. FunctionWrapper                 -> Generates polymorphic proxy trampolines around entry points (argument XOR shuffling, frame depth mutation, return masking)
 3. FunctionCallObfuscate (FCO)     -> Eliminates direct imports via dlopen/dlsym runtime resolution inside callers and proxies
 4. AntiDebugging                   -> Injects ptrace, hardware breakpoint (DR0-7), TF single-step probes, violent exit handler
 5. StringEncryption                -> Encrypts global strings with GF(2^8) stubs & injects volatile exit zeroizers
 6. ConstantEncryption (Phase 1)    -> Encrypts original programmer literals before CFG transformations
 7. Per-Function Transformation Loop:
    ├── 7a. Substitution (Sub)      -> Arithmetic expansions (x + y -> algebraic identities)
    ├── 7b. MBA Obfuscation         -> Multivariate non-linear boolean-arithmetic + polymorphic hardware barriers
    ├── 7c. Split Basic Blocks      -> Slices basic blocks, cutting MBA expressions across blocks + stack confusion
    ├── 7d. Bogus Control Flow (BCF)-> Injects opaque hardware predicates & clones split blocks into loops
    ├── 7e. Chaos State Machine (CSM)-> Replaces CFG topology with logistic-map quadratic chaotic dispatch
    ├── 7f. Classic Flattening (CFF)-> Fallback CFF with branchless algebraic masking for functions skipped by CSM
    └── 7g. Vector Obfuscation (Vec)-> Lifts remaining scalar logic & dispatch state into SIMD vector space
 8. ConstantEncryption (Phase 2)    -> Encrypts state constants & jump keys generated by BCF/CSM/CFF
 9. IndirectBranch                  -> Encrypts jump targets via Knuth multiplicative hashing into jump tables
10. Cleanup Markers                 -> Erases temporary compiler sentinel declarations (ensia_*) before symbol scrambling
11. FeatureElimination              -> Strips DWARF metadata, anonymizes TU path to "a", drops llvm.ident, clears COMDATs, internalizes ODR linkages, scrambles private/internal symbols (_f<hex>, _v<hex>, _a<hex>)
12. LTO Evasion                     -> Stamps functions with Attribute::OptimizeNone and Attribute::NoInline

Hardened Anti-Reverse Engineering Defenses

1. Branchless Algebraic State Transitions (Zero-SPOF CFF/CSM)

Traditional CFF relies on store volatile on a switch state variable. Once an attacker strips volatile, compiler passes (opt -O3, sccp, simplifycfg) unflatten the switch instantly. Ensia implements pure branchless algebraic masking:

$$\begin{aligned} \text{condExt} &= \text{zext}_{i1 \to i32}(\text{cond}) \in \{0, 1\} \\\ \text{mask} &= 0 - \text{condExt} \in \{0, \text{0xFFFFFFFF}\} \\\ \text{diff} &= \text{caseTrue} \oplus \text{caseFalse} \\\ \text{nextState} &= \text{caseFalse} \oplus (\text{mask} \land \text{diff}) \\\ \text{opaqueState} &= \text{insertOpaqueBarrier}(\text{nextState}) \end{aligned}$$
// Bitwise equivalent (Zero-SPOF branchless state transition in LLVM IR):
uint32_t condExt     = (uint32_t)cond;                         // 0 or 1
uint32_t mask        = 0 - condExt;                            // 0x00000000 or 0xFFFFFFFF
uint32_t diff        = caseTrue ^ caseFalse;
uint32_t nextState   = caseFalse ^ (mask & diff);              // branchless state select
uint32_t opaqueState = insertOpaqueBarrier(nextState);         // polymorphic hardware barrier

Even with all volatile keywords maliciously stripped from the IR, opt -O3 fails to eliminate the dispatch switch (100% switch retention).

2. Polymorphic Hardware Barriers

The static asm sideeffect "xorb $0, $0" signature has been replaced by a randomized 9-variant x86 instruction family (orb, andb, addb, subb, rolb, rorb, incb/decb, notb/notb) and 4 ARM64 pipeline barriers (prfm pldl1keep, prfm pstl1keep, prfm pldl2keep, isb sy, dmb ishld). Uniform signature and YARA rule detection is completely defeated.

3. 3-Tier Anti-Taint Engine

  • Tier 1 (Global Identity LUT): Dereferences bytes through indirect memory pointers (BytePtr = GEP(LUT, BarrieredByte); load volatile), severing register-level ALU dataflow edges.
  • Tier 2 (Implicit Control-Flow Bit Laundering): Reconstructs all 8 bits using select(bit_test, 1 << bit, 0) over pure constants. Because Dynamic Taint Analysis engines do not propagate taint across constant control dependencies, taint tags are sanitized.
  • Tier 3 (SIMD Vector Diffusion): Packs values into <4 x i32> / <2 x i64> vectors, mixes with entropy tokens, permutes lanes via shufflevector, and unpacks under polymorphic barriers, defeating scalar register trackers (Triton / angr).

4. String Encryption Anti-Dump

Decrypted strings do not persist in memory. At function returns (ReturnInst) and exception unwinds (ResumeInst), an inlined isVolatile memset zeroizes the plaintext buffer, and the atomic status lock is reset (release ordering). Memory snapshots taken post-execution reveal only \x00\x00....

5. AntiHook Dual-Defense & In-Flight Auditing

Ensia deploys a multi-tier hook detection and memory integrity defense architecture:

  • Entry Prologue Guard: Embedded directly in the function entry block, inspecting the function start against classic and modern hook sequences: 1-byte opcodes (0xE9 JMP rel32, 0xEB short JMP, 0xCC INT3, 0x68 PUSH imm32, 0xF1 ICEBP), 2-byte opcodes (0x48 0xB8 MOVABS RAX, 0x8B 0xFF MOV EDI,EDI hotpatch, 0x90 0x90 double NOP sled, 0x0F 0x0B UD2, 0xCD 0x03 INT 3), 64-bit RIP-relative indirect jumps (0xFF 0x25 JMP [RIP+disp32]), and AArch64 branches (0x14000001 B .+4, BRK, LDR x16/x17).
  • Scattered In-Flight CFG Auditing: Injects random runtime audits across non-EH interior basic blocks. If an attacker uses a trampoline hook that bypasses the entry guard and jumps past the prologue, executing interior basic blocks re-audits the function head (F[0], F[1]). All check sites branch to a single consolidated cold handler with violent exit, eliminating register pressure and preserving LLVM exception-handling invariants.
  • Embedded Integrity Self-Check with Data-Flow Entanglement (check_integrity): Cryptographic checksums of code pages are coupled directly into function execution tokens (T_env / T_exp). Any in-memory tampering or breakpoint patching silently corrupts downstream program state, avoiding obvious exception breakpoints.
  • Direct Syscall Bypass (direct_syscall): Bypasses userland libc hooks via direct kernel system calls (svc #0 / syscall).

Empirical Benchmark & Resilience Metrics

Tested against aggressive deobfuscation pipelines (opt -passes='default<O3>' and opt -passes='sccp,simplifycfg,instcombine,dce,gvn'):

Obfuscation Pass Test Symbol Expansion opt -O3 Retention Stripped-Barrier Retention SMT / Z3 Resistance
Substitution (SUB) test_arithmetic 5.8x 100.0% 100.0% Medium
MBA Obfuscation (MBA) test_arithmetic 15.5x 98.2% 71.0% (6x over base) Exponential (>50,000x)
String Encryption (STR) test_strings 556.0x 85.4% 74.4% High
Basic Block Split (SPLIT) test_sequential_math 1.7x 100.0% 100.0% Structural
Bogus Control Flow (BCF) test_control_flow 31.6x 97.5% 97.5% High
Control Flow Flat (CFF) test_control_flow 7.8x 92.6% 92.6% (Zero switches lost) High
Chaos State Machine (CSM) test_control_flow 21.1x 97.9% 97.9% Very High
Constant Encryption (CONST) test_constants 32.0x 80.7% 63.0% (20x over base) Very High (Feistel + adb.tok)
Indirect Branch (INDIBR) test_control_flow 12.5x 89.5% 84.6% (19/19 indirectbr) High
Vector Obfuscation (VOBF) test_sequential_math 11.8x 92.1% 92.1% (374/374 vectors) Vector Taint Diffusion
Full Combined Pipeline All 14 passes 393.6x 97.1% 100.0% (aggr) Exponential Solver Explosion

Usage & Integration

1. Clang C / C++ Integration

# 1. Automatic configuration discovery (loads ./ensia.toml if present in current directory):
clang -fpass-plugin=/path/to/libEnsia.so -O2 main.c -o main

# 2. Explicit configuration via environment variable:
ENSIA_CONFIG=/path/to/ensia.toml \
clang -fpass-plugin=/path/to/libEnsia.so -O2 main.c -o main

# 3. Quick preset selection via environment variable:
ENSIA_PRESET=mid \
clang -fpass-plugin=/path/to/libEnsia.so -O2 main.c -o main

# 4. Frontend plugin invocation with -mllvm option parsing:
clang -Xclang -load -Xclang /path/to/libEnsia.so \
  -mllvm -ensia-config=ensia.toml \
  -O2 main.c -o main

2. Rust (cargo / rustc) Integration

# Build binary crate with Ensia Rust plugin (auto-discovers ./ensia.toml):
ENSIA_CONFIG=ensia.toml RUSTC_BOOTSTRAP=1 \
RUSTFLAGS="-Z llvm-plugins=/path/to/libEnsia_rust.so -C passes=ensia" \
cargo build --release

3. Presets Overview

Preset Flag / Env Included Passes & Characteristics Recommended Use Case
low -enable-lowobf
ENSIA_PRESET=low
Sub + MBA + Split + BCF + StrEnc + ConstEnc. Minimal code bloat, fast compile. Debugging, rapid testing, performance-critical modules.
mid (Recommended) -enable-medobf
ENSIA_PRESET=mid
Sub + MBA + Split + BCF + ConstEnc + StrEnc + Flatten + Vec + IndirBranch. Balanced production protection. Production releases, commercial SDKs, game protection.
high -enable-highobf
ENSIA_PRESET=high
All passes active at high intensity. CSM preferred over Flatten, Feistel tier active, AntiHook, AntiDebug, FCO, FunctionWrapper. Core financial assets, licensing engines, critical algorithms.
max -enable-maxobf
ENSIA_PRESET=max
All passes at maximum intensity: BCF prob=100 loop=3, CSM nested 2-level dispatch, Vec 512-bit, ConstEnc kshare=6 + Feistel, FW 3 rounds, violent exit. Red-team deliverables, stress-testing toolchains.

4. Structured TOML Configuration (ensia.toml)

Search order: -mllvm -ensia-config=<path> > ENSIA_CONFIG=<path> > ./ensia.toml.

[global]
preset = "mid"            # "low" | "mid" (recommended) | "high" | "max" | "csm_vec"
verbose = false           # Print transformation logs to stderr
trace = false             # Emit step-by-step scheduler checkpoints
demangle_names = true     # Demangle C++ / Rust function symbols in logs

# ── 15 Configurable Passes (supports canonical names & short aliases) ──────────
[passes.bcf]
enabled = true
probability = 60          # Block selection probability (0–100)
iterations = 1           # BCF loop iterations (1–5)
complexity = 4           # Opaque predicate depth (1–10)
entropy_chain = true     # Chain hardware predicates (CPUID, RDTSC/CNTPCT)
junk_asm = true          # Inject polymorphic hardware inline-ASM barriers
junk_asm_min = 2
junk_asm_max = 6

[passes.constant_encryption]  # alias: [passes.const_enc]
enabled = true
iterations = 1
share_count = 3          # Bivariate MBA additive split shares (2–8)
feistel = true           # 4-round non-linear Feistel cipher network
substitute_xor = true    # MBA expansion of recombination XORs
force_value = ["^0x9E3779B9$", "^0xDEADBEEF$"]
skip_value = ["^0x0$", "^0x1$"]

[passes.string_encryption]    # alias: [passes.str_enc]
enabled = true
probability = 100
anti_dump = true          # Volatile memory zeroization at function exits
force_content = [".*key.*", ".*secret.*", ".*token.*"]
skip_content = ["^%[0-9]*[a-zA-Z]$", "^PASS$", "^FAIL$"]

[passes.substitution]         # alias: [passes.sub]
enabled = true
probability = 60
iterations = 1

[passes.mba]
enabled = true
probability = 50
layers = 2
heuristic = true         # Zero-noise identity verification

[passes.split_blocks]         # alias: [passes.split]
enabled = true
splits = 3               # Slice points per basic block
stack_confusion = true   # Push/pop or str/ldr stack desynchronization

[passes.chaos_state_machine]  # alias: [passes.csm]
enabled = true
warmup = 128             # Q32 logistic map transient warmup iterations
nested_dispatch = false  # Hierarchical 2-level cluster dispatch
max_blocks = 5000

[passes.flattening]           # alias: [passes.cff]
enabled = false          # Fallback CFF with branchless algebraic masking

[passes.vector_obfuscation]   # alias: [passes.vec]
enabled = true
probability = 40
width = 128              # 128 (SSE/NEON), 256 (AVX2), 512 (AVX-512)
shuffle = true           # Random bijective shufflevector permutations
lift_comparisons = true

[passes.indirect_branch]      # alias: [passes.indir]
enabled = true
use_stack = true
enc_jump_target = true   # Knuth multiplicative modular inverse hashing

[passes.function_wrapper]     # alias: [passes.fw]
enabled = false
probability = 50
times = 1

[passes.function_call_obfuscate] # alias: [passes.fco]
enabled = false
flag = 0
symbol_config_path = ""  # Path to external symbol import JSON config

[passes.anti_hooking]         # alias: [passes.anti_hook]
enabled = true
inline_x86 = true        # E9 / 48 B8 prologue integrity scans
inline_aarch64 = true    # 0x14000001 (B .+4) inline hook scans
inline_win = true        # Windows API inline hook detection
direct_syscall = true    # Direct kernel syscall bypass (svc #0 / syscall)
antirebind = true        # Counter dynamic linker symbol rebinding
check_integrity = true   # Embedded code segment cryptographic self-checks
precompiled_ir_path = "" # Path to external precompiled LLVM IR defense stubs

[passes.anti_debugging]       # alias: [passes.anti_dbg]
enabled = true
probability = 80         # Hardware debug registers DR0-7, EFLAGS.TF single-step probes
precompiled_ir_path = "" # Path to external precompiled LLVM IR defense stubs

[passes.anti_class_dump]      # alias: [passes.anti_acd]
enabled = false          # Objective-C / Swift metadata scrambling (macOS / iOS)
use_initialize = true
rename_methodimp = true
scramble_methods = true
dummy_selectors = false
dummy_count = 8
encrypt_strings = true   # Dynamic stack string decryption
anti_hook = true         # Objective-C runtime hook detection
opaque_barriers = true   # Memory barriers on runtime pointers

# ── Granular per-module / per-function regex policy overrides ──────────────────
[[policy]]
module_regex = ".*crypto.*"
function_regex = ".*(encrypt|decrypt|sign|verify).*"
preset = "high"
passes.chaos_state_machine.nested_dispatch = true
passes.constant_encryption.feistel = true
passes.constant_encryption.share_count = 4
passes.mba.layers = 3
passes.anti_debugging.probability = 100
passes.anti_hooking.direct_syscall = true

[[policy]]
module_regex = ".*"
function_regex = "^(main|fast_path_.*)$"
passes.bcf.enabled = false
passes.chaos_state_machine.enabled = false
passes.flattening.enabled = false

Building From Source

Prerequisites

  • CMake 3.20+
  • Ninja or Make
  • LLVM & Clang (version 21, 22, or 23)
  • Rust toolchain (optional, for libEnsia_rust.so and web)

Build Steps (Linux / macOS)

git clone https://github.com/Apich-Organization/ensia.git
cd ensia
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

Outputs:

  • build/obfuscation/libEnsia.so (Clang plugin)
  • build/obfuscation/libEnsia_rust.so (Rustc plugin)

Build Steps (Windows with MSVC / clang-cl)

mkdir build && cd build
cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_DIR="C:/path/to/llvm/lib/cmake/llvm"
ninja Ensia

To run the automated verification suite:

bash test/run_obf_tests.sh

Licensing & Attribution

This project is licensed under the AGPL-3.0. It includes code and concepts continuing the lineage of Hikari and LLVM. See LEGAL.md for full details on project history and original authors.

Sponsorship & Funding Policy

We welcome sponsorships supporting open-source compiler security research. Please review our Sponsorship Policy for details on fund allocation, contribution options via Open Collective, and corporate tiers.

Code of Conduct & Security

Please consult CODE_OF_CONDUCT.md and SECURITY.md for vulnerability reporting guidelines and ethical standards.

Releases

Packages

Used by

Contributors

Languages