Engine Architecture

Engine Architecture

Authoritative reference for ZEngine's overall architecture — threads, communication, and how all subsystems fit together. Update it when a significant architectural change lands in develop.

See also: Rendering Domain · Memory Management · Asset Manager

---

Table of Contents

• Big Picture
• Thread Model
• Main Thread — Per-Frame Work
• Render Thread — Per-Frame Work
• Cross-Thread Communication
• ECS and Simulation
• Asset Pipeline
• Virtual File System (VFS)
• Initialization Order
• Shutdown Order

---

Big Picture

flowchart TD
    main["Main Thread\nECS simulation\nInput / VFS tick\nImportCoordinator\nApp::Update / UI"]
    render["Render Thread\nRRM::BeginFrame\nRenderGraph::Execute\nSwapchain present\nRRM::EndFrame"]
    scheduler["MainThreadScheduler\n512 MPSC slots"]
    pool["ThreadPool workers\nGltfImporter\nAssimpImporter\nImportCoordinator jobs"]

    main -->|"mailbox write (lock-free)"| render
    pool -->|"Post(ctx, fn)"| scheduler
    scheduler -->|"Drain() — step 9 each frame"| main
    pool -->|"IngestMesh / IngestTextures"| main

Loading

Key design rules:

• Main thread owns all ECS state, input, and UI logic.
• Render thread owns all Vulkan command recording and submission.
• Neither thread blocks on the other within a frame.
• Background workers never touch Vulkan — GPU work is enqueued to RenderResourceManager and drained on the render thread.

---

Thread Model

Thread	Count	Launched by	Joins at
Main thread	1	OS entrypoint (main())	Process exit
Render thread	1	Engine::Run()	Engine::Deinitialize()
ThreadPool workers	hw_concurrency - 1	ThreadPoolHelper::Initialize()	ThreadPoolHelper::Shutdown()

The render thread is launched before MainThreadRun() begins and joined first in teardown — before any GPU resource is touched.

---

Main Thread — Per-Frame Work

Engine::MainThreadRun() — ZEngine/ZEngine/Engine.cpp

loop (until s_close_requested):

  1. PollEvent()                    GLFW platform event pump
  2. VFSContext::Tick()             Drain FileWatcher debounce queue
  3. skip frame if minimized
  4. frame_timer.End()             Measure raw delta (clamped 250 ms)
  5. accumulator.Accumulate(dt)

  6. Fixed simulation (60 Hz, max 5 catch-up steps):
       WorldTick::Tick(scene, fixed_dt, cmds)   ECS systems in DAG order
       WorldCommands::Flush(scene)              Deferred structural mutations
       ActorManager::Tick(fixed_dt)             Actor OnTick() callbacks
       Scene::SnapshotTransforms()              PreviousPosition ← Position
       accumulator.ConsumeStep()

  7. alpha = accumulator.Alpha()    Interpolation factor for renderer

  8. ImportCoordinator::Tick()      Dispatch pending import jobs to ThreadPool

  9. MainThreadScheduler::Drain()   Execute callbacks posted by background threads

 10. g_app->Update(raw_dt)         Editor/game non-ECS update, layer updates

 11. Build RenderPayload            PrepareScene + OnRenderUI (ImGui)

 12. Mailbox write (lock-free)      Push payload to render thread

 13. FrameRateCap::Wait()          vsync-off only: cap at 300 FPS

---

Render Thread — Per-Frame Work

Engine::RenderThreadRun() — ZEngine/ZEngine/Engine.cpp

loop (until s_request_terminate):

  1. Mailbox read (lock-free)         Wait for payload; re-use last if none

  2. Swapchain::AcquireNextImage      Block on fence for in-flight slot

  3. RRM::BeginFrame(frame_index)
       FlushPendingUploads:
         ├─ ResetGeometryBuffersInternal  (if scene-reload flag is set)
         ├─ BeginBatchUpload
         ├─ DoUploadMesh × N           All mesh uploads in ONE GPU submission
         ├─ EndBatchUpload
         └─ DoUploadTexture × M        Per-texture timeline upload

  4. AppRenderPipeline::Tick (if InstancesDirty)
       GetMeshOffsets → SubMeshAllocation per submesh
       Upload TransformSB, DrawDataSB, VkDrawIndirectCommand[]

  5. RenderGraph::Execute
       DepthPrePass  → DrawIndirect   (all scene meshes, depth only)
       SkyboxPass    → DrawIndexed    (builtin cube)
       GridPass      → DrawIndexed    (builtin quad + push constants)
       GbufferPass   → DrawIndirect   (all scene meshes, full G-buffer)
       LightingPass  → Draw(3)        (full-screen deferred lighting triangle)

  6. ImGuiRenderer::Render            Upload vertex/index, record ImGui draw commands

  7. Swapchain::Present               Submit + present, advance timeline semaphore

  8. RRM::EndFrame(frame_index)       Drain DeferredFreeQueue for this slot

The render thread never writes ECS state. It reads only from the RenderPayload and RenderScene::MeshInstance[] (seqlock snapshot).

---

Cross-Thread Communication

Three distinct channels, each with a different mechanism.

1. Main → Render: Lock-free mailbox

sequenceDiagram
    participant Main as Main Thread
    participant Ring as RenderPayload[3] ring buffer
    participant Render as Render Thread

    Main->>Ring: PrepareScene + OnRenderUI → RenderPayload[next]
    Main->>Ring: MailBoxBufferHead.store(next, release)
    Render->>Ring: tail = MailBoxBufferHead.load(acquire)
    alt head != tail
        Render->>Ring: consume payload[tail]
    else head == tail
        Render->>Render: re-use last payload (drop frame)
    end

Loading

MailBoxBufferHead is a PaddedAtomic<uint32_t> index into a 3-slot ring. No mutex, no blocking.

2. Background → Main: MainThreadScheduler

sequenceDiagram
    participant Worker as ThreadPool Worker
    participant MTS as MainThreadScheduler
    participant Main as Main Thread (step 9)

    Worker->>MTS: Post(ctx, fn) — fetch_add claims slot
    Worker->>MTS: write ctx + fn
    Worker->>MTS: ready[slot].store(true, release)
    Main->>MTS: Drain() — exchange write_cursor to 0
    loop for each claimed slot
        Main->>MTS: spin on ready[i].load(acquire)
        Main->>Main: fn(ctx)
        Main->>MTS: ready[i].store(false)
    end

Loading

512 arena-allocated slots, lock-free MPSC, no heap, C-style Post(void* ctx, void (*fn)(void*)).

Current users: AssetImporterUIComponent — posts TriggerScan() after import completes or fails.

3. Asset thread → Render thread: RRM pending queue

sequenceDiagram
    participant Asset as Asset Thread
    participant AM as AssetManager
    participant RRM as RenderResourceManager
    participant GPU as GPU

    Asset->>AM: IngestMesh(mesh, hier)
    AM->>AM: AssetRegistry::SetState(Loaded)
    AM->>RRM: OnAssetReady callback
    RRM->>RRM: m_pending.push (m_pending_mutex)
    Note over RRM: Next BeginFrame on render thread
    RRM->>RRM: FlushPendingUploads — drain m_pending[]
    RRM->>GPU: DoUploadMesh → vkCmdCopyBuffer

Loading

m_pending[MAX_PENDING=256] is a fixed array protected by m_pending_mutex. All Vulkan work stays on the render thread.

---

ECS and Simulation

Files: ZEngine/ZEngine/ECS/

Two-tier object model — see actor-ecs-architecture.md for full detail:

Tier	Type	Used for
1	Actor — C++ object with vtable, owns an EntityID	Player, camera, lights
2	Raw EntityID — data only, no object	Foliage, particles, projectiles

Both tiers share the same ComponentStorage<T> sparse-set arrays. Scene::ForEach<Ts...>() visits all alive entities regardless of tier.

WorldTick — System Scheduler

WorldTick builds a DAG from registered systems and runs them in topological wave order.

flowchart LR
    R["Register systems\nat startup"]
    C["Commit()\nbuild DAG\nassign waves"]
    W0["Wave 0 — single\nPhysicsSystem\ninline on main thread"]
    W1["Wave 1 — parallel\nRenderCullSystem\nAudioSystem\nThreadPool workers"]
    W2["Wave 2 — single\nParticleSpawnSystem\ninline on main thread"]
    Flush["WorldCommands::Flush\napply deferred mutations"]

    R --> C --> W0 --> W1 --> W2 --> Flush

Loading

WorldCommands — Deferred Mutations

Calling scene.AddComponent() from inside a parallel-wave system would race with other workers. WorldCommands defers all structural mutations to after the wave barrier.

flowchart TD
    PW["Parallel wave execution"]
    S0["worker 0\nstaging[0].SpawnEntity(...)"]
    S1["worker 1\nstaging[1].AddComponent(...)"]
    B["barrier — all workers done"]
    M0["main: commands.Merge(staging[0])\nfix up SpawnCallbackIndex offsets"]
    M1["main: commands.Merge(staging[1])"]
    F["WorldCommands::Flush → apply to scene"]

    PW --> S0 & S1 --> B --> M0 --> M1 --> F

Loading

Staging buffers are pre-allocated at Commit() (one per system, arena-backed) and reused every frame via Clear() — no heap after the first exercised frame.

---

Asset Pipeline

flowchart TD
    src["Source file\n.glb / .fbx / .png / .hdr"]
    importer["GltfImporter / AssimpImporter\n(ThreadPool worker via ImportCoordinator)"]
    cooked["Cooked artifacts\n.zemesh · .zematerial · Assets/Textures/…"]
    ingest["AssetManager::IngestMesh\nAssetManager::IngestTextures\nAssetManager::IngestMaterial"]
    registry["AssetRegistry::SetState(Loaded)\n→ RRM::OnAssetReady → m_pending.push"]
    rrm["RRM::FlushPendingUploads\n(render thread, next BeginFrame)"]
    gpu["GPU global VB/IB\nTextureArray (bindless)"]
    pipeline["AppRenderPipeline::Tick\n(when InstancesDirty)\nSubMeshAllocation + DrawIndirect"]

    src --> importer --> cooked --> ingest --> registry --> rrm --> gpu --> pipeline

Loading

On scene reload: EditorScene::ExtractAsync calls RRM::ResetGeometryBuffers() before ingestion; the render thread resets global buffer cursors to 0 so new geometry starts fresh.

---

Virtual File System (VFS)

Files: ZEngine/ZEngine/Core/VFS/

graph TD
    ctx["VFSContext"]
    m1["Mount /ZodiacEngine\n→ VFSDiskBackend\n→ cwd/ZodiacEngine/\npriority = -1"]
    m2["Mount /\n→ VFSDiskBackend\n→ project root\npriority = 0"]
    path["VFSPath\nnormalized, immutable\nno-alloc value type"]
    scanner["VFSScanner\nasync directory walker\npopulates content browser cache"]
    watcher["VFSFileWatcher\nFSEvents / inotify / RDCW\ndebounced via VFSContext::Tick()"]
    meta["MetaFileIO\n.meta sidecars\nuuid · source path · artifact path"]
    reg["AssetRegistry\nuuid → SlotHandle + state\ntriggers RRM::OnAssetReady on Loaded"]

    ctx --> m1 & m2
    ctx --> scanner & watcher & meta & reg
    ctx --> path

Loading

FileWatcher → hot-reload flow:

sequenceDiagram
    participant FW as VFSFileWatcher
    participant CTX as VFSContext::Tick (main thread)
    participant AR as AssetRegistry
    participant IC as ImportCoordinator
    participant TP as ThreadPool

    FW->>CTX: debounced change event
    CTX->>AR: SetStale(uuid)
    CTX->>IC: Enqueue(path, Immediate)
    IC->>TP: dispatch reimport job (next Tick)

Loading

---

Initialization Order

Engine::Initialize() — ZEngine/ZEngine/Engine.cpp

sequenceDiagram
    participant EP as EntryPoint
    participant Win as GameWindow
    participant Dev as VulkanDevice
    participant VFS as VFSContext
    participant AM as AssetManager
    participant ECS as ECS (Scene · ActorManager · WorldTick)
    participant IC as ImportCoordinator
    participant RRM as RenderResourceManager
    participant MTS as MainThreadScheduler

    EP->>Win: Initialize (GLFW + VkInstance + surface)
    Win->>Dev: Initialize (queues, VMA, bindless descriptors)
    Dev->>VFS: Initialize (mount table, disk backend)
    VFS->>AM: Initialize (UUID map, registry, 512 MB sub-arena)
    AM->>ECS: Initialize (Scene, ActorManager, WorldCommands, WorldTick)
    ECS->>IC: Initialize + register importers (Gltf, Assimp, EnvMap)
    IC->>RRM: Initialize (global VB/IB, texture timelines, upload pool)
    RRM->>AM: InitFallbackTexture (hot-pink 4×4 — requires RRM live)
    AM->>VFS: InitWatcher (FSEvents / inotify / RDCW)
    VFS->>MTS: Initialize (512 MPSC slots from MainArena)

Loading

Hard dependencies: Device before VFS (surface), RRM before fallback texture, watcher after working directory.

---

Shutdown Order

flowchart TD
    T["s_request_terminate = true"]
    RT["Join render thread\nNO GPU work after this"]
    MTS["MainThreadScheduler::Shutdown\ndiscard pending tasks"]
    ECS["ECS::ActorManager::Shutdown\nECS::Scene::Shutdown"]
    RRM["RRM::Shutdown\nQueueWaitAll · destroy upload pools · free global buffers"]
    AM["AssetManager::Shutdown"]
    ARP["AppRenderPipeline::Shutdown\nRenderGraph::Dispose (pipelines, framebuffers)\nImGuiRenderer::Deinitialize"]
    VFS["VFS::Shutdown"]
    DEV1["VulkanDevice::Deinitialize\nQueueWaitAll · first PendingFree drain\nSwapchainPtr→Dispose · CommandBufferMgr::Deinit\nsecond PendingFree drain"]
    WIN["Window::Deinitialize"]
    DEV2["VulkanDevice::Dispose\nfinal PendingFree drain\nGpuMem::Shutdown · vkDestroyDevice"]

    T --> RT --> MTS --> ECS --> RRM --> AM --> ARP --> VFS --> DEV1 --> WIN --> DEV2

Loading

See Rendering Domain — Shutdown for the Vulkan object destruction rules.