Skip to content

Engine Architecture

Jump to bottom
Jean Philippe edited this page Aug 18, 2026 · 5 revisions

Engine Architecture

Authoritative reference for ZEngine's overall architecture — threads, communication, and how all subsystems fit together. Update it when a significant architectural change lands in develop.

See also: Rendering Domain · Memory Management

Table of Contents

Big Picture

┌───────────────────────────────────────────────────────────────────┐
│                           Process                                 │
│                                                                   │
│  ┌──────────────────────┐  mailbox  ┌─────────────────────────┐  │
│  │     Main Thread      │ ────────► │     Render Thread       │  │
│  │                      │           │                         │  │
│  │  ECS simulation      │           │  RRM::BeginFrame        │  │
│  │  Input / VFS tick    │           │  RenderGraph::Execute   │  │
│  │  ImportCoordinator   │           │  Swapchain present      │  │
│  │  App::Update / UI    │           │  RRM::EndFrame          │  │
│  └──────────┬───────────┘           └─────────────────────────┘  │
│             │ Post()                                               │
│             ▼                                                      │
│  MainThreadScheduler ◄── ThreadPool workers (import, audio…)      │
│                                                                    │
│  ThreadPool workers ─────────────────────────────────────────────► │
│    GltfImporter, AssimpImporter, ImportCoordinator jobs            │
└───────────────────────────────────────────────────────────────────┘

Key design rules:

  • Main thread owns all ECS state, input, and UI logic.
  • Render thread owns all Vulkan command recording and submission.
  • Neither thread blocks on the other within a frame.
  • Background workers never touch Vulkan — GPU work is enqueued to RenderResourceManager and drained on the render thread.

Thread Model

Thread Count Launched by Joins at
Main thread 1 OS entrypoint (main()) Process exit
Render thread 1 Engine::Run() Engine::Deinitialize()
ThreadPool workers hw_concurrency - 1 ThreadPoolHelper::Initialize() ThreadPoolHelper::Shutdown()

The render thread is launched before MainThreadRun() begins and joined first in teardown — before any GPU resource is touched.

Main Thread — Per-Frame Work

Engine::MainThreadRun() — ZEngine/ZEngine/Engine.cpp

loop (until s_close_requested):

  1. PollEvent()                    GLFW platform event pump
  2. VFSContext::Tick()             Drain FileWatcher debounce queue
  3. skip frame if minimized
  4. frame_timer.End()             Measure raw delta (clamped 250 ms)
  5. accumulator.Accumulate(dt)

  6. Fixed simulation (60 Hz, max 5 catch-up steps):
       WorldTick::Tick(scene, fixed_dt, cmds)   ECS systems in DAG order
       WorldCommands::Flush(scene)              Deferred structural mutations
       ActorManager::Tick(fixed_dt)             Actor OnTick() callbacks
       Scene::SnapshotTransforms()              PreviousPosition ← Position
       accumulator.ConsumeStep()

  7. alpha = accumulator.Alpha()    Interpolation factor for renderer

  8. ImportCoordinator::Tick()      Dispatch pending import jobs to ThreadPool

  9. MainThreadScheduler::Drain()   Execute callbacks posted by background threads

 10. g_app->Update(raw_dt)         Editor/game non-ECS update, layer updates

 11. Build RenderPayload            PrepareScene + OnRenderUI (ImGui)

 12. Mailbox write (lock-free)      Push payload to render thread

 13. FrameRateCap::Wait()          vsync-off only: cap at 300 FPS

Render Thread — Per-Frame Work

Engine::RenderThreadRun() — ZEngine/ZEngine/Engine.cpp

loop (until s_request_terminate):

  1. Mailbox read (lock-free)         Wait for payload; re-use last if none

  2. Swapchain::AcquireNextImage      Block on fence for in-flight slot

  3. RRM::BeginFrame(frame_index)
       FlushPendingUploads:
         ├─ ResetGeometryBuffersInternal  (if scene-reload flag is set)
         ├─ BeginBatchUpload
         ├─ DoUploadMesh × N           All mesh uploads in ONE GPU submission
         ├─ EndBatchUpload
         └─ DoUploadTexture × M        Per-texture timeline upload

  4. AppRenderPipeline::Tick (if InstancesDirty)
       GetMeshOffsets → SubMeshAllocation per submesh
       Upload TransformSB, DrawDataSB, VkDrawIndirectCommand[]

  5. RenderGraph::Execute
       DepthPrePass  → DrawIndirect   (all scene meshes, depth only)
       SkyboxPass    → DrawIndexed    (builtin cube — disabled if no env map)
       GridPass      → DrawIndexed    (builtin quad + push constants)
       GbufferPass   → DrawIndirect   (all scene meshes, full G-buffer)
       LightingPass  → Draw(3)        (full-screen deferred lighting triangle)

  6. ImGuiRenderer::Render            Upload vertex/index, record ImGui draw commands

  7. Swapchain::Present               Submit + present, advance timeline semaphore

  8. RRM::EndFrame(frame_index)       Drain DeferredFreeQueue for this slot

The render thread never writes ECS state. It reads only from the RenderPayload and RenderScene::MeshInstance[] (seqlock snapshot).

Cross-Thread Communication

Three distinct channels, each with a different mechanism.

1. Main → Render: Lock-free mailbox

Main thread                              Render thread
─────────────────────────────────────   ───────────────────────────────────
PrepareScene(r_payload)
OnRenderUI(r_payload)
MailBoxBufferHead.store(next, release) ─► tail = MailBoxBufferHead.load(acquire)
                                         if head != tail: consume payload

MailBoxBufferHead is a PaddedAtomic<uint32_t> index into a 3-slot ring buffer. Main writes with store(release); render reads with load(acquire). If head == tail the render thread re-uses the last payload (drops frame). No mutex, no blocking.

2. Background → Main: MainThreadScheduler

ThreadPool worker (any thread):         Main thread (step 9 each frame):
  MainThreadScheduler::Post(ctx, fn)    MainThreadScheduler::Drain()
  ──── lock-free MPSC slot-claim ────►  execute all committed tasks sequentially

Lock-free MPSC: PaddedAtomic<uint32_t> write cursor claims a slot via fetch_add; data is written; per-slot PaddedAtomic<bool> ready published with release ordering. Drain atomically swaps the cursor to 0 and spins (acquire) on each slot's ready flag — terminates immediately in practice since producers complete before the next frame boundary.

512 arena-allocated slots, no heap, no std::mutex, C-style Post(void* ctx, void (*fn)(void*)).

Current users:

  • AssetImporterUIComponent — posts TriggerScan() after import completes or fails

Pattern for new callers:

MainThreadScheduler::Post(this, [](void* ctx) {
    static_cast<MyComponent*>(ctx)->OnWorkDone();
});

3. Asset thread → Render thread: RRM pending queue

Asset thread:                            Render thread (BeginFrame):
  AssetManager::IngestMesh(mesh)
  AssetRegistry::SetState(Loaded)
  → RRM::OnAssetReady callback
  → m_pending.push  (m_pending_mutex) ─► FlushPendingUploads drains m_pending[]
                                         → DoUploadMesh → GPU copy via m_upload_cmd

m_pending[MAX_PENDING=256] is a fixed array protected by m_pending_mutex. Asset threads write; the render thread drains each BeginFrame. All Vulkan work stays on the render thread.

ECS and Simulation

Files: ZEngine/ZEngine/ECS/

Two-tier object model — see actor-ecs-architecture.md for full detail:

Tier Type Used for
1 Actor — C++ object with vtable, owns an EntityID Player, camera, lights
2 Raw EntityID — data only, no object Foliage, particles, projectiles

Both tiers share the same ComponentStorage<T> sparse-set arrays. Scene::ForEach<Ts...>() visits all alive entities regardless of tier.

Simulation order each fixed step (step 6 of main loop):

WorldTick::Tick       — ECS systems in DAG order; parallel wave if no write conflicts
WorldCommands::Flush  — deferred spawn/destroy/add-component applied atomically
ActorManager::Tick    — Actor::OnTick() on all live actors
Scene::SnapshotTransforms  — PreviousPosition ← Position (for interpolation)

Not yet wired (issue #604): Scene::FillRenderableTransforms() is implemented but never called — ECS transforms do not flow to the GPU TransformSB. The renderer currently reads transforms from RenderScene::MeshInstance[].

Asset Pipeline

Source file (.glb / .fbx / .png / .hdr)
    │
    ▼  ImportCoordinator::Tick() → ThreadPool worker
GltfImporter / AssimpImporter / EnvironmentMapImporter
    Cook → .zemesh + .zematerial (JSON) + Assets/Textures/…
    No Vulkan, pure CPU / file I/O
    │
    ▼  AssetManager::IngestMesh / IngestTextures
    CPU-side: copy to arena arrays, register in AssetRegistry
    SetState(Loaded) → RRM::OnAssetReady → m_pending.push
    │
    ▼  (next RRM::BeginFrame on render thread)
FlushPendingUploads → BatchUpload → GPU global VB/IB
    │
    ▼  AppRenderPipeline::Tick (when InstancesDirty)
GetMeshOffsets → SubMeshAllocation → DrawIndirect

On scene reload: EditorScene::ExtractAsync calls RRM::ResetGeometryBuffers() (atomic flag) before ingestion; the render thread resets global buffer cursors to 0 so the new scene's geometry starts fresh. See Rendering Domain — Global Geometry Buffers.

Virtual File System (VFS)

Files: ZEngine/ZEngine/Core/VFS/

All engine and project assets are accessed through VFS paths regardless of physical location.

VFSContext
├── Mount("/ZodiacEngine", VFSDiskBackend → cwd/ZodiacEngine/)  priority=-1
└── Mount("/",             VFSDiskBackend → project root)        priority=0
Component Purpose
VFSPath Normalized, immutable, no-alloc path value type (/-separated)
VFSScanner Async directory walker; populates content browser cache
VFSFileWatcher Debounced platform watcher (FSEvents / inotify / RDCW); fired via VFSContext::Tick() on the main thread
MetaFileIO Reads/writes .meta sidecars (stable UUID, source path, artifact path)
AssetRegistry UUID-indexed store; triggers RRM::OnAssetReady on Loaded state

FileWatcher → hot-reload flow:

VFSFileWatcher → debounce → VFSContext::Tick() (main thread)
  → AssetRegistry::SetStale(uuid)
  → ImportCoordinator::Enqueue(path, Immediate)
  → ImportCoordinator::Tick() dispatches reimport job to ThreadPool

Initialization Order

Engine::Initialize() — ZEngine/ZEngine/Engine.cpp

 1. Validate preconditions   (MemoryManager, Logger, ThreadPool — caller responsibility)
 2. Arena-allocate EngineContext
 3. GameWindow::Initialize   (GLFW + Vulkan surface + VkInstance + VkDevice)
 4. VulkanDevice::Initialize (queues, command pools, VMA allocator, bindless descriptor sets)
 5. VFSContext::Initialize   (mount table, VFSDiskBackend for engine assets)
 6. AssetManager::Initialize (UUID map, registry, 78 MB sub-arena)
 7. InputManager::Initialize
 8. ECS: Scene, ActorManager, WorldCommands, WorldTick::Initialize
 9. ImportCoordinator::Initialize + register importers (Gltf, Assimp, EnvMap)
10. RenderResourceManager::Initialize  (global VB/IB, texture timelines, upload pool)
11. AssetManager::InitFallbackTexture  (hot-pink 4×4 PNG — requires RRM live)
12. VFSContext::InitWatcher            (FSEvents/inotify/RDCW)
13. GLFW scroll callback
14. MainThreadScheduler::Initialize   (512 MPSC slots from MainArena)
15. g_app->CurrentWindow = window

Hard dependencies: Device before VFS (surface), RRM before fallback texture, watcher after working directory.

Shutdown Order

 1. s_request_terminate = true           Signal render loop to exit
 2. g_render_thread.join()              Wait for render thread — NO GPU work after this
 3. MainThreadScheduler::Shutdown()     Discard any pending main-thread tasks
 4. ECS::ActorManager::Shutdown()
 5. ECS::Scene::Shutdown()
 6. RRM::Shutdown()                     QueueWaitAll, destroy upload pools, free buffers
 7. AssetManager::Shutdown()
 8. AppRenderPipeline::Shutdown()       RenderGraph::Dispose (pipelines, framebuffers)
 9. VFS::Shutdown()
10. VulkanDevice::Deinitialize()        Drain deferred-free queue, destroy Vulkan objects
11. Window::Deinitialize()
12. VulkanDevice::Dispose()            Final PendingFree drain, GpuMem shutdown, vkDestroyDevice

See Rendering Domain — Shutdown for the Vulkan object destruction rules.

Clone this wiki locally