daslang -exe: 3-tier shared-module resolution (exe_dir → das_root → absolute) by borisbat · Pull Request #2579 · GaijinEntertainment/daScript

borisbat · 2026-05-05T14:34:49Z

Summary

Standalone executables built with daslang -exe now resolve dynamic shared modules at startup by trying:

<exe_dir>/<rel_path> — daspkg release bundles (modules sit next to exe)
<das_root>/<rel_path> — SDK install AND local dev build
<fallback_abs_path> — baked-at-codegen absolute (legacy fallback)

Previously only tier 3 existed: the absolute path captured at compile time was baked verbatim into the binary. This worked for in-place dev builds (the path stays valid as long as the exe doesn't move) but broke two real scenarios that ship today:

Scenario	Layout	Before	After
Local dev build	exe at `<repo>/bin/<cfg>/`, modules at `<repo>/modules/`	✅ baked absolute	✅ tier 2 (`getDasRoot()` strips `bin/<cfg>` → `<repo>`)
CMake install / CI SDK download	exe at `<install>/bin/`, modules at `<install>/modules/`	❌ baked absolute → build-machine FS	✅ tier 2 (`<install>/modules/...`)
daspkg release bundle	exe at `<bundle>/`, modules at `<bundle>/modules/`	❌ same	✅ tier 1 (`<bundle>/modules/...`)

This is the prerequisite for the upcoming daspkg release command (PR #2 of the daspkg release plan) and also fixes the SDK-install case independently.

Implementation

src/builtin/module_jit.cpp — new jit_register_dynamic_module_resolve(rel_path, fallback_abs_path, mod_name) runtime helper. Plus test seams: jit_set_exe_file_for_test_ and jit_set_path_exists_for_test_ so the resolution logic is unit-testable without dlopening anything.
modules/dasLLVM/daslib/llvm_exe.das — inject_main emits the new helper instead of jit_register_dynamic_module. New private compute_modules_relative_suffix extracts the modules/<X>/... suffix at codegen time via to_generic_path + substring search (cross-platform; never scans for both /X/ and \X\).

Test plan

tests-cpp/small/test_jit_module_resolve.cpp — 7 cases covering resolution priority, _debug-variant detection, Windows backslash exe paths, empty-input edge cases (uses test seams to drive logic with synthetic exe paths and a mock filesystem predicate).
tests/exe-paths/_uses_sqlite.das + run_layouts.cmake — end-to-end integration: builds a standalone exe via daslang -exe, deploys it to bundle / sdk-install / local-dev layouts in tmp dirs, runs each from a neutral cwd. Labeled small so CI's ctest -L small step (build.yml:340/343) picks it up across the full platform matrix.
Existing utils (aot.exe, daspkg.exe, benchctl.exe, mcp.exe — all built via daslang -exe) re-run cleanly after the change. Verified locally.
dastest --test tests/: 7822 tests, 7816 passed, 0 failed, 0 errors, 6 skipped.
test_aot --use-aot --test tests/: 7216 tests, 7210 passed, 0 failed, 0 errors, 6 skipped.

Doc note

skills/filesystem.md — clarifies that the existing "never rfind('/') / rfind('\\\\') on paths" rule has one legitimate exception: searching for a named component (e.g. /modules/, /.git/) after to_generic_path normalize. Single-line addition + one row in the "pick the right tool" table.

🤖 Generated with Claude Code

Copilot

Pull request overview

This PR updates daslang -exe standalone executables to resolve dynamic shared modules at startup via a 3-tier search order (<exe_dir> → <das_root> → baked absolute fallback), enabling relocatable bundles/SDK installs while preserving legacy behavior.

Changes:

Add a runtime resolver helper in the JIT runtime to choose the best dynamic-module path before loading.
Update LLVM standalone-exe codegen to emit relative modules/... paths plus a baked absolute fallback.
Add both C++ unit tests and a CMake-driven integration test to validate resolution across real deployment layouts; update filesystem path-handling guidance.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`src/builtin/module_jit.cpp`	Adds resolver logic + test seams; new API `jit_register_dynamic_module_resolve`.
`modules/dasLLVM/daslib/llvm_exe.das`	Emits rel-path + fallback-abs registration for dynamic modules in standalone exes.
`tests-cpp/small/test_jit_module_resolve.cpp`	Unit tests for resolution priority, `_debug` variant behavior, and Windows-style exe paths.
`tests/exe-paths/run_layouts.cmake`	Integration test driver that builds and runs a standalone exe across multiple layouts.
`tests/exe-paths/CMakeLists.txt`	Registers the integration test with CTest and labels it `small`.
`tests/exe-paths/_uses_sqlite.das`	Minimal standalone program requiring `dasSQLITE` to validate module loading.
`skills/filesystem.md`	Documents the approved exception for component search after `to_generic_path` normalization.
`CMakeLists.txt`	Adds the new `tests/exe-paths` subdirectory to the test build.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…bsolute) Standalone executables built with `daslang -exe` now resolve dynamic shared modules at startup by trying, in order: 1. <exe_dir>/<rel_path> — daspkg release bundles (modules sit next to exe) 2. <das_root>/<rel_path> — SDK install AND local dev build 3. <fallback_abs_path> — baked-at-codegen absolute (legacy fallback) Previously only tier 3 existed: the absolute path captured at compile time was baked verbatim into the binary, which broke two scenarios shipping today — CMake-installed / CI-released SDKs (paths point to the build machine's filesystem) and copied bundles (same problem). This change fixes both without regressing in-place dev builds. inject_main computes the rel_path suffix from the last `/modules/` segment via to_generic_path + substring search; runtime helper `jit_register_dynamic_module_resolve` does the tiered lookup with file- existence probes. Tests: - tests-cpp/small/test_jit_module_resolve.cpp — 7 cases covering resolution priority, _debug-variant detection, Windows backslash exe paths, empty-input edge cases (uses test seams to drive the logic with synthetic exe paths and a mock filesystem predicate). - tests/exe-paths/_uses_sqlite.das + run_layouts.cmake — end-to-end integration: builds a standalone exe via `daslang -exe`, deploys it to bundle / sdk-install / local-dev layouts, runs each from a neutral cwd. Labeled `small` so CI's `ctest -L small` picks it up. skills/filesystem.md — clarifies that the "never rfind('/') / rfind('\\\\') on paths" rule has one legitimate exception: searching for a named component (`/modules/`, `/.git/`) after a `to_generic_path` normalize. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

`daspkg release` produces <out>/<bundle_name>/ containing the standalone exe, every .shared_module dylib the program transitively requires, every asset matching the project's release_include globs, AND every transitive dep package's release_include assets shipped at <bundle>/modules/<DepName>/. The bundle runs with no daslang installed (PR #2579 — exe-relative shared modules — is the prerequisite). `release()` hook in `.das_package`: - release_main(script) entry .das (required for the project) - release_name(name) override bundle name - release_include(glob) ship matching assets - release_exclude(glob) skip matching assets - release_shared_module(name) force-include a dylib not auto-detected (matches by daslang-side name OR package directory name) Auto-detection: Step 1, in `daslang -exe`: a new `-list-shared-modules <path>` flag causes llvm_exe.das to write a JSON `ReleaseDeps` file. The writer walks `program_for_each_module(prog)` and emits two parallel lists: * `shared_modules`: registered dylibs whose daslang-side name appears in the program (deduped by absolute path — one .shared_module can host several daslang modules, e.g. dasStbImage hosts stbimage, raster, stbtruetype). * `das_modules`: program modules whose source `.das` file walks up to a `.das_package` ancestor (within 8 levels). The walk-up filter drops daslib stdlib (no .das_package above) and SDK-shipped modules under `<das_root>/modules/<X>/` (also no .das_package), keeping only daspkg-installed deps. Each entry is tagged with `package_dir`. Argv parsing uses `daslib/clargs.find_flag_raw_value`. JSON via `daslib/json_boost.sprint_json`. No new C++ policy field; main.cpp gains a 4-line consume-arg stub matching the existing -das-profiler-log-file pattern. Step 2, in `cmd_release`: `from_JV` the JSON, ship dylibs at the bundle-relative path the exe expects, then walk unique `package_dir` entries (skipping the project root) and run each dep's `release()` to copy its `release_include` files to <bundle>/modules/<DepName>/<rel>. False-positive tests cover both kinds: - test_cmd_release_no_false_positive_shared: a registered-but-unused dylib must NOT appear in the bundle (verifies registry-walk filter). - test_cmd_release_no_false_positive_das: a project with no daspkg deps creates no <bundle>/modules/ tree (verifies that daslib stdlib + project-local .das files don't show up). - test_cmd_release_transitive_dep: synthetic dep with its own .das_package + release_include — its assets land at the expected relative path; excluded files don't ship. Plus end-to-end tests test_cmd_release_pure_daslang and test_cmd_release_native_dep (gated on dasSQLITE registration). End-to-end verified against examples/games/sequence with dasCards installed (`bin/daslang utils/daspkg/main.das -- install --root examples/games/sequence`): 6 shared-module dylibs auto-detected (dasGlfw, dasStbImage, dasLiveHost, dasAudio, dasHV, dasPUGIXML), plus cards/svg-cards.svg + cards/DejaVuSerif.ttf shipped via the dep's release(). 199/199 daspkg tests pass; tests/ regression 7816/7822 pass (6 pre-existing skips, 0 failed); AOT regression 7210/7216 pass. Sweep: release() added to 10 in-repo .das_package files (real apps + daspkg fixtures); new .das_package + release() for examples/games/{ arcanoid,asteroids,river_run} so all 4 games are releasable. Library packages under examples/daspkg/packages/ deliberately left without release() — they're modules to be installed, not standalone projects. skills/daspkg.md gains a Release section covering the hook, auto-detection, output layout, transitive traversal, and a runtime-asset-path caveat: `get_module_file_name` returns the build-machine path (not exe-relative), so deps that ship runtime assets still need exe-relative lookup themselves until daslang grows the equivalent of PR #2579 for that helper. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings May 5, 2026 14:34

Copilot started reviewing on behalf of borisbat May 5, 2026 14:35 View session