sync: OpenKotOR master (GLES + modawan integration)#167
Open
th3w1zard1 wants to merge 69 commits into
Open
Conversation
Co-authored-by: Copilot <copilot@github.com>
…sserts KotOR.js ships only engine/site assets and requires a local install (README: no game files in the repo). Reone's multi-GB artifact is engine.data from CMake embedded preload when REONE_WEB_GAME_DIR is set; default remains FS Access only. - Keep Emscripten main-loop unwind fix and SDL wheel switch cleanup. - Gate -sASSERTIONS=2 behind REONE_WEB_DEBUG_RUNTIME (default OFF). - Gitignore local game trees, smoke PNG, and .cursor/ so copyrighted assets are never committed.
Serve KotOR mirror via game-manifest before FS picker; bind dev server to 127.0.0.1. Raise web wasm heap cap for MEMFS mirrors; smoke script spawns serve and waits for menu log. Skip glGenerateMipmap on Emscripten (texture + PBR cube-array); tighten Emscripten engine loop/audio/context paths.
fix(web): WebGL2 GLES fixes, optional cube-map-array, KotOR.js-style FS glue
- Introduced compiler cache support for faster builds using sccache or ccache. - Updated GitHub Actions workflow to include additional integration tests and upload engine.data as part of the WASM artifacts. - Improved server response handling and added directory indexing for easier navigation of served files. - Added verification script to ensure completeness of the WASM bundle after builds. This commit enhances the development experience and ensures a more robust build and serve process for the WASM engine.
Revert the mistaken local upstream merge; document that modawan#111 remains an open draft. Run verify_wasm_bundle after the wasm job build, trigger workflow on cursor/* pushes, and treat engine.data as optional in the verifier when only the glsl preload package is present.
Local Release wasm build verified (engine links, verify_wasm_bundle passes). Set CMP0167 OLD for FindBoost on CMake 3.30+ and pin setup-emsdk to 5.0.7 to match the toolchain used for reproduction.
Merge serve-smoke and wasm build into one ubuntu-latest job, disable cancel-in-progress on concurrency, add engine.html HTTP smoke after link, and set explicit contents:read permissions.
Provides a local parity path when hosted runners are queued.
serve.py no longer requires engine artifacts in --directory when --game-root is set and engine.html is absent (matches workflow step). Restore cancel-in-progress concurrency so stuck queued runs do not block later pushes indefinitely.
Queued jobs were waiting 30+ minutes with no steps; removing concurrency avoids cancelled/pending runs holding the group. Local ci_build_wasm.sh remains the parity gate until a hosted run completes.
Multiple parallel queued runs were competing for org runners. Serialize per ref and cancel superseded pushes so the latest commit gets a slot.
Avoid EADDRINUSE when 28765/28766 are taken after a prior serve. Deepen wasm CI plan with org runner-queue findings; cancel stale master workflows to unblock Build WASM assignment.
Skip wasm workflow on docs-only pushes; document runner_id=0 queue diagnosis and local ci_build_wasm.sh parity in doc/webassembly.md.
Prevents upload-artifact failures once hosted runners assign. R3 verified locally (verify_wasm_bundle + serve smoke); R1 still blocked on org queue.
Document force-cancel and workflow_dispatch recovery for runner_id 0 stalls. Operational dispatch attempted; org runner assignment still pending.
* feat(dataminer): migrate generators to extract::Installation Replace legacy KeyBif/Folder/RIM/ERF containers in dataminer with the Installation index API, add shared lookup helpers, and fix TSL nwscript loading to use the TSL chitin path instead of KotOR. * fix(ci): use ResRef field in dataminer 2DA push_back ResRef is a struct member on ResourceId, not a callable; fixes dataminer build on Linux and Windows CI. * fix(ci): ResRef field access in dataminer models lookup * docs: LFG pass 18 — dataminer migration plan update Record stuck wasm-ci recovery and merge-ready state for PR #5. * docs: LFG pass 19 — wasm runner online, re-trigger CI Record self-hosted runner recovery and pending wasm-ci after 45m queue stall. --------- Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Replace pre-merge pass 19 draft with final landed state after PR #5 merge.
Mark superseded WASM menu/playwright plans completed, sync Installation port deferred notes after dataminer merge, record pass 21 smoke gates.
Squash merge glad-gles: GLES renderer, cubemap IBL fallback, GUI preview lighting, StreamMusic search, headless CI smokes, wasm-ci on ubuntu-latest.
Sync pass 5–7 plan artifacts after squash merge of PR #7. Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
* [game] Deserialize StaticCamera directly from GFF * [game] Deserialize Sound directly from GFF * [game] Load Force Power selection metadata * [game] Add Force Power display entries * [gui] Render level-up Force Powers with IconChain * [gui] Select level-up Force Powers Implements level-up Force Powers selection. This adds the full Force Powers level-up stack: - loads selection-relevant `spells.2da` metadata - loads class power-gain data from `classpowergain.2da` - builds spell display entries from prerequisite chains - renders the Force Powers picker with the shared `IconChain` control - supports known/selectable/locked/chosen visual states - supports focus/name/description display - requires spending granted power selections before advancing - applies selected powers through the temporary level-up character attributes before Finish The implementation follows the level-up Feats architecture from #136, but keeps spell-specific data/rules separate. Force Power chains are built from `spells.2da` prerequisites, not Feats successor logic, `pips`, or `masterspell`. Notes ----- - No Feats behavior is changed. - No actionbar Force Power behavior is changed. - No droid-specific picker behavior is claimed. - `IconChain` changes are additive generic APIs used for focus/state refresh. Known bugs ---------- - 'Feats' doesn't illuminate the tile of the next valid feat if the preceding feat is chosen and prereqs are satisfied for the next feat - Both feats and powers exhibit a bug where the feat/power name reverts to 'description' if scrolling after highlighting a feat/power * [game] Deserialize Encounter directly from GFF * [game] Deserialize Store directly from GFF * [graphics] Replace GLAD wrapper from GL to GLES2 * Workarounds * [game] Route looted items through inventory insertion (#156) * Routes droppable loot transfers through the normal inventory insertion path. * Stacks inventory items by tag instead of blueprint resref. * Merges full incoming stack counts when transferring stacked items. * Registers usable item metadata when newly inserting items into creature inventory. * Removes stale source item attributes when moving droppable creature inventory. ## Notes Looted items previously bypassed the normal `addItem` path and were pushed directly into the destination inventory list. That meant they could appear in inventory without going through the same stacking and usable-item metadata registration as items granted through `additem`. Droppable semantics are not loosened; creature inventory still preserves authored `Dropable` values, and non-droppable creature inventory remains excluded from corpse loot. * [gui] Wire action cycling arrows (#157) * Wires red target-action frame arrow bands to cycle target actions. * Renders target-action arrows only when a slot has multiple actions. * Wires bottom-right action bar arrows to cycle instead of using the item/action. * Shows bottom-right action bar arrows only for multi-action categories. * Preserves center-click queueing/use of the displayed action. * Wraps cycling at the ends of action lists. * [game] Deserialize StaticCamera directly from GFF * [game] Deserialize Sound directly from GFF * [game] Fix selection of a dialog owner in StartConversationAction Problem: ------- There are 3 cases that we need to handle: 1. Scripts use ActionStartConversation with DialogResRef. 2. Scripts use ActionStartConversation *without* DialogResRef (empty string). 2.1. Caller has a non-empty conversation field. 2.2. Caller has an empty conversation field. 3. Player initiates a dialog with a Creature or Placeable. In case (1) the dialog should start for the caller, and use the provided DialogResRef. Most scripts follow (2.1) and we should select the caller as the dialog owner. However, Taris m02aa entry trigger (Duros and Sith) seems to rely on (2.2): - Trigger `sithtalk` calls StartConversationAction, but the object does not have Conversation field, and the script doesn't specify DialogResRef. - ObjectToConverse is a Placeable `trooper1_invis` with Conversation `tar02_preraid`. Here we need to select `trooper1_invis` as the dialog owner and start `tar02_preraid` dialog. Player actions (3) should always use ObjectToConverse Conversation and set this object as the dialog owner - not the player. Implementation: -------------- ActionStartConversation sets a default value of DialogResRef to caller->conversation() before constructing StartConversationAction object. Once we get to StartConversationAction::execute, DialogResRef is not empty, even if the script did not specify it. This way (1) and (2.1) are equivalent in StartConversationAction::execute: both have DialogResRef set, and both need to select the actor as the dialog owner. Before the patch, the engine used to set DialogResRef to ObjectToConverse Conversation for player actions, but that would make case (3) indistinguishable from (1) and (2.1), while require the opposite behavior. The patch changes player actions to never pass DialogResRef parameter. Therefore cases (2.2) and (3) are equivalent now. In both cases, we need to select the ObjectToConverse as the owner, and use its Conversation parameter. * [game] Deserialize Encounter directly from GFF * [game] Deserialize Store directly from GFF * [game] Change Party to keep available NPC as Creatures (#160) Party used to only contain blueprints (templates) for NPC, and spawn new creatures when an NPC is requested. That means if an NPC is removed from the party, it used to be created from scratch next time this NPC is needed. This approach is not going to work for savegames, because they contain fully instantiated creatures. The patch replaces template references with Creature objects. It is still possible to add available members by templates, but they are instantiated immediately. * fix(graphics): GLES IBL fallback via per-probe cubemap pool (Option A) Replace the 2D-array pseudo-cubemap fallback with real GL_TEXTURE_CUBE_MAP pairs sampled through a switch in the PBR combine pass. Keep OES cube map array as the 16-layer fast path when the driver exposes it. Cap fallback derived layers at four to stay within ES fragment texture unit limits. Includes shadow geometry-shader workaround and updated GLES plan. * docs(plan): record GLES Option A validation status Document landed work, partial runtime checks, and follow-ups for the cubemap-pool IBL fallback on glad-gles. * fix(graphics): restore GLES menu rendering and shader compilation on Mesa Main menu draws and the cursor were black on GLES because BGR/BGRA uploads used invalid glTexImage2D format enums; swizzle to RGB/RGBA before upload. Also harden GLSL for the stricter Mesa ES compiler so the PBR pipeline builds. * ci: add GLES Linux build workflow and smoke validation scripts Build the GLES engine on Ubuntu with Mesa dev packages, pack shaders, and provide X11 smoke scripts for main-menu and optional dev-warp checks. * fix(ci): run GLES build on ubuntu 25.10 container libsdl3-dev is unavailable on the ubuntu-24.04 runner image; match the existing Linux workflow container so dependency install succeeds. * [game][gui] Route K2 in-game menu Party and Messages (#158) Kotor 2 used to display the 'message' page instead of the party page. The following addresses that core issue and several other adjacent ones: - Routes K2’s Party top-bar/HUD icon to hosted Party Selection. - Keeps the K2 in-game menu top tabs visible while Party Selection is active. - Wires K2 Journal’s “Messages Log” button to open Messages. - Makes K2 Messages Close return to Journal. - Preserves K1 `BTN_MSG` Messages behavior and standalone Party Selection paths. * fix(scene): use retro renderer for orthographic GUI 3D views on GLES PBR deferred combine fails to light main-menu Malak (orthographic camera) while the retro forward path renders correctly with pbr=1 enabled. Route orthographic scene graphs to retro and clear shadow uniforms when no shadow light is active. * docs(plan): update GLES menu validation status and next steps Record orthographic retro-renderer fix and gate warp smoke on user menu sign-off. * fix(gles): restore main menu 3D lighting and add headless smoke harness SceneInitializer no longer zeroes scene ambient unless explicitly set; default mesh material colors when MDL ambient/diffuse are zero; set main menu ambient to 0.5. Smoke scripts run on isolated Xvfb + Mesa llvmpipe so validation never touches the user's Wayland session and can run in CI. * docs(plan): mark GLES main menu character fix completed * fix(gles): add warp smoke via console commands and CI headless selftest Use commands-file warp for reliable headless module loads on Xvfb instead of xdotool GUI clicks, and gate GLES CI with a no-game Xvfb sanity check. * fix(gles): harden menu smoke startup and update IBL plan status Align menu validation with warp smoke process hygiene (cwd-scoped kill, startup log gate, window focus) and record landed headless smoke slice. * docs(plan): record green OpenKotOR GLES CI on 0f94b03 Note successful push workflow including headless selftest and defer master merge until GLES conflict resolution. * docs(plan): note master merge and green CI on b638139 * fix(resource): resolve theme music from StreamMusic in sound search AudioClips switched to soundSearchOrder() during the Installation port, but that order only searched StreamSounds. Add Music and Voice so mus_theme_cult and similar streamed WAV/MP3 clips load again. * test(gles): assert main menu theme audio loads in menu smoke Fail the headless menu smoke if mus_theme* clips fail to decode after the StreamMusic search-order fix; update the GLES plan with CI head. * docs(plan): record real-GPU menu smoke and Malak lighting gap Log and screenshot smokes pass on DISPLAY=:0; document that the main menu 3D preview body is still a near-black silhouette on real GPU. * fix(scene): restore GUI preview lighting for GLES main menu GUI ortho scenes now keep all embedded MDL lights active, skip area lightmaps and env/bump maps that suppress world ambient, and use full world ambient. Menu smoke adds a Malak-region luminance regression gate. * docs(plan): add GLES merge-ready LFG completion plan * docs(plan): mark GLES LFG merge-ready plan completed * docs(plan): record GLES LFG verification pass 2 Re-validated headless, menu (Malak mean ~960), and warp smokes on fadd5a6. * docs(plan): link upstream OpenKotOR PR #7 for GLES merge * fix(ci): stop canceling self-hosted wasm builds on PR sync Long wasm-ci jobs on the self-hosted runner were aborted when concurrent PR events stacked. Disable cancel-in-progress and use shallow checkout. * fix(ci): run wasm-ci on ubuntu-latest after self-hosted auth failure Self-hosted wasm-ci runner checkout fails with git auth errors; use GitHub-hosted ubuntu-latest with apt deps until runner credentials are fixed. * docs(plan): note wasm-ci green on ubuntu-latest for PR #7 * docs(plan): record squash merge of OpenKotOR PR #7 to master * docs(plan): record downstream modawan PR #163 handoff (pass 6) * docs(plan): add GLES LFG pass 7 sync plan on glad-gles * fix(ci): run wasm-ci on master after GLES merge Post-GLES master never triggered build-wasm.yml (cursor/** only). Enable master pushes, refresh progress.md, harden smoke stdout when piped, and record local menu + end_m01aa warp validation. --------- Co-authored-by: modawan <modawan42@proton.me> Co-authored-by: Eldbury <Leneldbury@gmail.com> Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
* chore(resource): remove unused KeyBif and Folder containers No callers remain after the Installation extract port; drop dead IResourceContainer implementations and CMake entries. * docs: LFG pass 25 — record dead container removal progress * docs: LFG pass 26 — archaeology findings and Phase B scope Record infer-intent analysis: only ErfResourceContainer has a caller; Rim/Exe/Memory are orphaned. Defer saveload migration to Phase B. --------- Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
peekSavedGame never called ErfResourceContainer::init(), so save-list metadata could not load. Read savenfo/screen directly with ErfReader and remove the orphaned IResourceContainer stack (Erf/Rim/Exe/Memory). Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Remove unused IResourceContainer header left after #11, document that cursor/web-wasm-gles-and-fs-access is superseded by master, and point CI runbook dispatch at master. Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Check scene node type before static_pointer_cast to ModelSceneNode. Fixes undefined behavior when dialog targets lack a model scene node. Ported from modawan#159. Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
…real GPU (#15) Strict drivers rejected compressed-texture mipmap generation, stale MRT draw buffer state, and OIT routing for GUI meshes with alpha-channel textures. Smoke scripts now target build-gles/bin and gate on the Malak torso region. Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
* [game] Load Force Power selection metadata * [game] Add Force Power display entries * [gui] Render level-up Force Powers with IconChain * [gui] Select level-up Force Powers Implements level-up Force Powers selection. This adds the full Force Powers level-up stack: - loads selection-relevant `spells.2da` metadata - loads class power-gain data from `classpowergain.2da` - builds spell display entries from prerequisite chains - renders the Force Powers picker with the shared `IconChain` control - supports known/selectable/locked/chosen visual states - supports focus/name/description display - requires spending granted power selections before advancing - applies selected powers through the temporary level-up character attributes before Finish The implementation follows the level-up Feats architecture from #136, but keeps spell-specific data/rules separate. Force Power chains are built from `spells.2da` prerequisites, not Feats successor logic, `pips`, or `masterspell`. Notes ----- - No Feats behavior is changed. - No actionbar Force Power behavior is changed. - No droid-specific picker behavior is claimed. - `IconChain` changes are additive generic APIs used for focus/state refresh. Known bugs ---------- - 'Feats' doesn't illuminate the tile of the next valid feat if the preceding feat is chosen and prereqs are satisfied for the next feat - Both feats and powers exhibit a bug where the feat/power name reverts to 'description' if scrolling after highlighting a feat/power * [game] Route looted items through inventory insertion (#156) * Routes droppable loot transfers through the normal inventory insertion path. * Stacks inventory items by tag instead of blueprint resref. * Merges full incoming stack counts when transferring stacked items. * Registers usable item metadata when newly inserting items into creature inventory. * Removes stale source item attributes when moving droppable creature inventory. ## Notes Looted items previously bypassed the normal `addItem` path and were pushed directly into the destination inventory list. That meant they could appear in inventory without going through the same stacking and usable-item metadata registration as items granted through `additem`. Droppable semantics are not loosened; creature inventory still preserves authored `Dropable` values, and non-droppable creature inventory remains excluded from corpse loot. * [gui] Wire action cycling arrows (#157) * Wires red target-action frame arrow bands to cycle target actions. * Renders target-action arrows only when a slot has multiple actions. * Wires bottom-right action bar arrows to cycle instead of using the item/action. * Shows bottom-right action bar arrows only for multi-action categories. * Preserves center-click queueing/use of the displayed action. * Wraps cycling at the ends of action lists. * [game] Deserialize StaticCamera directly from GFF * [game] Deserialize Sound directly from GFF * [game] Fix selection of a dialog owner in StartConversationAction Problem: ------- There are 3 cases that we need to handle: 1. Scripts use ActionStartConversation with DialogResRef. 2. Scripts use ActionStartConversation *without* DialogResRef (empty string). 2.1. Caller has a non-empty conversation field. 2.2. Caller has an empty conversation field. 3. Player initiates a dialog with a Creature or Placeable. In case (1) the dialog should start for the caller, and use the provided DialogResRef. Most scripts follow (2.1) and we should select the caller as the dialog owner. However, Taris m02aa entry trigger (Duros and Sith) seems to rely on (2.2): - Trigger `sithtalk` calls StartConversationAction, but the object does not have Conversation field, and the script doesn't specify DialogResRef. - ObjectToConverse is a Placeable `trooper1_invis` with Conversation `tar02_preraid`. Here we need to select `trooper1_invis` as the dialog owner and start `tar02_preraid` dialog. Player actions (3) should always use ObjectToConverse Conversation and set this object as the dialog owner - not the player. Implementation: -------------- ActionStartConversation sets a default value of DialogResRef to caller->conversation() before constructing StartConversationAction object. Once we get to StartConversationAction::execute, DialogResRef is not empty, even if the script did not specify it. This way (1) and (2.1) are equivalent in StartConversationAction::execute: both have DialogResRef set, and both need to select the actor as the dialog owner. Before the patch, the engine used to set DialogResRef to ObjectToConverse Conversation for player actions, but that would make case (3) indistinguishable from (1) and (2.1), while require the opposite behavior. The patch changes player actions to never pass DialogResRef parameter. Therefore cases (2.2) and (3) are equivalent now. In both cases, we need to select the ObjectToConverse as the owner, and use its Conversation parameter. * [game] Deserialize Encounter directly from GFF * [game] Deserialize Store directly from GFF * [game] Change Party to keep available NPC as Creatures (#160) Party used to only contain blueprints (templates) for NPC, and spawn new creatures when an NPC is requested. That means if an NPC is removed from the party, it used to be created from scratch next time this NPC is needed. This approach is not going to work for savegames, because they contain fully instantiated creatures. The patch replaces template references with Creature objects. It is still possible to add available members by templates, but they are instantiated immediately. * [game][gui] Route K2 in-game menu Party and Messages (#158) Kotor 2 used to display the 'message' page instead of the party page. The following addresses that core issue and several other adjacent ones: - Routes K2’s Party top-bar/HUD icon to hosted Party Selection. - Keeps the K2 in-game menu top tabs visible while Party Selection is active. - Wires K2 Journal’s “Messages Log” button to open Messages. - Makes K2 Messages Close return to Journal. - Preserves K1 `BTN_MSG` Messages behavior and standalone Party Selection paths. * [nfc][game] Refactor deserialization of GVT to a separate function * [game] Deserialize player inventory `inventory.res` is a top-level GFF in a save game container. It lists all items that the player (party) has. The player character (Creature) has no items in the corresponding Creature GFF. Reone treats the player character as "the party" for inventory logic. Items are always transferred to the player character, inventory screens list items from the player character, etc. The patch follows this logic and deserializes `inventory.res` into the player character's inventory. * [resource] Handle LocString without a localization index Some mods do not set localization index for names and descriptions. Use the reference instead of displaying an empty string. --------- Co-authored-by: Eldbury <Leneldbury@gmail.com> Co-authored-by: modawan <modawan42@proton.me> Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Re-integrate modawan master after OpenKotOR #18 squash; keeps extract stack and modawan save-game / LocString fixes.
Author
|
Ready for maintainer merge (2026-06-05): OpenKotOR:glad-gles @ a27107b — MERGEABLE. ctest + headless menu smoke pass. |
Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Author
|
Tracking sync (2026-06-05): |
Author
Maintainer merge checklist (2026-06-05)Status: ready to squash-merge.
Action: Squash merge this PR into modawan Handoff doc on OpenKotOR: |
Co-authored-by: Copilot <th3w1zard1@users.noreply.github.com>
Keep DialogGUI null-model guard from OpenKotOR #14 when resolving conflict.
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces closed #163 (closed without merge @ 2026-06-05). Tracks OpenKotOR/reone
master@23a518fe:Test plan
master@23a518fectest+ headless GLES menu smoke (Malak torso mean ~6376)masterSupersedes