test(suite): document deliberate non-goals in lua53 suite#216
Merged
Conversation
Split the lua53 skipped-tests list into 'missing features' (still addressable) and 'deferred permanent' (deliberate non-goals). The four files in the latter group — main.lua, files.lua, attrib.lua, verybig.lua — exercise shell-out, file I/O, or filesystem-backed require semantics that conflict with this library's role as a sandboxed embedded Lua VM. Per-file rationale now lives inline in test/lua53_suite_test.exs and in ROADMAP's Deferred section. No behavioural change. Suite count unchanged (5/29 ready, 24 skipped). Full mix test: 1585 tests, 0 failures, 31 skipped. Plan: A20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Triage cluster — sandbox-refusal suite files
Plan:
.agents/plans/A20-triage-sandbox-refusals.mdGoal
Decide, per file, whether each of
main.lua,verybig.lua,files.lua,attrib.luais a permanent non-goal (skip with explanatory tag) or whetherthe suite runner should be configured to stub
os.execute/os.tmpname/os.getenvso the file completes. Produce per-file follow-up plans or skipannotations.
Decision: all four are skip-permanently. Each exercises capabilities
that conflict with this library's role as a sandboxed embedded Lua VM.
Per-file decisions
main.luaos.execute() is sandboxedluaas a subprocess. We are an embedded VM with no shell-out.files.luaos.getenv(_) is sandboxedio.*is a stub by design.attrib.luamodule 'io' not foundrequiresemantics that depend on writing files to disk and dynamically loading them.verybig.luaos.tmpname() is sandboxedos.tmpname()+io.output()+dofile()harness. The VM behaviour is interesting; the harness is not. A future plan could stub these for the suite runner only — not in scope here.Success criteria
skip-permanently; reasons live in
test/lua53_suite_test.exs@deferred_permanentmap.test/lua53_suite_test.exsgains a comment alongside the@tag :skipfor each of the four, and this is reflected inCHANGELOG.md(Unreleased) andROADMAP.md(Deferred section).verybig.luacould justify a future stub-for-suite plan, but it is recorded as a
possibility, not a commitment.)
mix testreports 1585 tests, 0 failures, 31 skipped(same as pre-flight). Suite test reports 29 tests, 0 failures, 24
skipped (5 ready / 4 deferred-permanent / 20 missing-feature).
Changes
Key change in
test/lua53_suite_test.exs: split the single@skipped_testslist into
@skipped_tests(missing features, future plans welcome) and@deferred_permanent(a%{file => reason}map for deliberate non-goals).The deferred-permanent tests get their own describe block with the reason
embedded in the test name and a
@tag :deferred_permanentfor filtering.Discoveries
os.execute()/os.tmpname()/os.getenv(_)was correct for three of them.attrib.luais a different sandbox policy: it dies onrequire"io"because there is no
ioglobal to cache. The cluster framing stillholds — all four exercise capabilities that conflict with our sandbox
role — but the failure mechanism for
attrib.luais package/loader-driven, not
os.*-driven.verybig.luais the most interesting of the four. The first ~140 linestest genuinely useful VM behaviour (RK opcodes, >64K constants), but
they live inside a generated Lua program written to a temp file and
executed with
dofile(). A future plan could stub these primitives inthe suite runner only to unlock that coverage. Recorded as a
possibility, not a commitment.
what's already in
ROADMAP.md's "Deferred" list. Updated ROADMAP andCHANGELOG instead.
Verification
Out of scope (intentional)
os.executeetc. for general consumption.os.tmpname/io.output/dofilefor the suite runner —recorded as a possible future plan, not shipped here.