Agent/stabilize foundation - #10
Open
FuryBaM wants to merge 313 commits into
Open
Conversation
…work stubs for WebAssembly
…d sockets and echo server
…er pool integration
…andling, and service tests
…egration - Implemented browser host mocks for HTTP and TCP in tools/absolute-wasm-browser-host.js and .mjs. - Added optional wasi-sysroot bootstrap script for Windows to facilitate libc experiments. - Updated documentation to reflect changes in WASM target capabilities and browser integration. - Introduced new CMake file for discovering wasi-sysroot paths. - Enhanced examples and tests to utilize new browser host features and mock functionalities. - Improved logging and error handling in the browser host environment.
- Implemented a static file server (scripts/serve-wasm-demo.mjs) to serve WASM demos with necessary cross-origin policies for SharedArrayBuffer. - Created a CMake script (tests/run-wasm-browser-session.cmake) to build and run a browser session for testing WASM modules. - Developed a main-thread client (tools/absolute-wasm-browser-session-client.js) for managing WASM sessions and handling logs. - Introduced a worker script (tools/absolute-wasm-browser-session-worker.js) to run WASM modules off the UI thread, supporting TCP operations. - Added a WebSocket-backed TCP worker (tools/absolute-wasm-ws-tcp-worker.js) to facilitate communication over WebSockets for nested workers.
- Updated loader.js to include taskPoolMode and sharedMemory in logs. - Enhanced bootstrap-wasi-sysroot.ps1 to download and extract compiler-rt builtins for selective wasi-libc kits. - Modified run-wasm-browser-task-pool.cmake to validate shared-memory task pool wiring in session workers. - Added run-wasm-shared-tasks.cmake to test shared-memory module with in-place task pool. - Introduced run-wasm-wasi-libc.cmake to validate coexistence of Absolute WASI runtime with selective wasi-libc objects. - Created wasi-libc-probe.c to provide a C probe for testing selective wasi-libc kits. - Updated absolute-wasm-browser-session-worker.js to support shared memory imports and task pool mode. - Implemented absolute-wasm-browser-shared-task-worker.js for shared-memory task execution in the browser. - Enhanced absolute-wasm-host.js to support shared memory in task pools and improved task management. - Created absolute-wasm-shared-task-worker.js for Node.js to handle shared-memory tasks without context copying.
…nds in developer CLI
…ilities - Implemented keyboard and mouse input APIs with edge detection (keyPressed/keyReleased, mousePressed/mouseReleased). - Added 2D drawing functions: drawLine, fillCircle, and blit for software rendering. - Updated examples to demonstrate new features: added a Pong game and enhanced the window example with delta time handling. - Improved README documentation to reflect new APIs and usage instructions. - Refactored runtime code to support input state tracking and frame timing.
… soft sprite functionality - Added gamepad input support using XInput for Windows, including connection status, button states, and axis values. - Implemented text input handling, allowing for Unicode code points to be captured and processed. - Introduced soft sprite functionality with the ability to load BMP images, clear, fill, and draw sprites. - Updated README documentation to reflect new input methods and sprite capabilities. - Refactored sprite management into a separate file for better organization and maintainability. - Ensured compatibility with headless and X11 runtimes by providing stubs for new features.
…pabilities - Implemented soft bitmap font rendering in the desktop plugin, allowing for text drawing using a built-in 8x8 ASCII font. - Added functions for measuring text width and height, as well as drawing text directly onto windows and sprites. - Created a new example (`text.abs`) demonstrating the usage of the soft font for HUD, multi-line text, and live typing. - Updated documentation to reflect new text rendering features and API changes.
…with strided blits
- Implemented a minimal OpenGL rendering hardware interface (RHI) for desktop platforms using WGL and OpenGL 3.3 core. - Added functions for GPU resource management including creation, destruction, and drawing operations. - Introduced a GPU class in the desktop plugin to manage OpenGL context and resources. - Enhanced the desktop runtime to provide native window handles for GPU backends. - Updated the manifest to include GPU support.
…th shaders, vertex buffers, and layouts - Updated README.md to reflect new GPU API structure and usage patterns. - Modified absolute-desktop.editor.json to provide accurate documentation and snippets for new GPU methods. - Refactored desktop_gpu_gl.cpp to implement a new frame model with beginFrame, endFrame, and pipeline binding. - Introduced VertexLayout, GpuShader, GpuBuffer, and GpuPipeline classes to encapsulate GPU resource management. - Updated Gpu class methods to support new pipeline-based rendering workflow. - Removed deprecated drawDemoTriangle method and integrated drawing logic into the new pipeline structure. - Enhanced error handling and validation across GPU resource creation and usage.
…nd related functionality
…mark runner Explicitly specialized generic calls rejected the implicit numeric conversions that ordinary overloads accept: SelectOverload seeded the substitution map from the explicit type arguments and then unified those same parameters against the argument types, so identity<int64>(7) failed with "no overload of 'identity' accepts (int32)" even though the non-generic int64 twice(int64) accepts the same literal. Explicit type arguments already bind every generic parameter, so skip unification in that case and let ConversionCost judge the arguments, exactly as it does for a non-generic candidate. Invalid calls are still rejected. The X11 desktop runtime declared a helper named KeyCode, which is ambiguous with the KeyCode typedef from X11/X.h and broke the Linux build whenever the X11 backend was selected. Rename it to MapKeyCode. Add benchmarks/run-benchmarks.py, a POSIX driver for the array, pointer-object, algorithm, collection, and value-ref suites. The existing run.bat/run-benchmark.ps1 drivers need MSVC and a WSL hop for the Absolute side, so none of the suites could run on Linux. The runner reproduces the same methodology: Absolute through absolutec --emit-llvm optimized by clang -O3 -march=native, checksum validation before a time is accepted, rotating language order, medians for optimized languages and separate Python sampling. Toolchains that are absent are skipped with a note instead of aborting the run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Running the suites on Linux leaves extension-less native executables in collection-suite/.benchmark-build/native/ and timestamped result files, neither of which belongs in the repository. The suite already tracks its Windows .exe/.obj/.ll/.class artifacts, so ignore that directory with negations that keep those visible, and ignore Linux result files by name across every suite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
An unqualified call inside an interface default method body reached the implicit-this path, which only searched classes and structs. The interface matched neither, so every such call failed codegen with "missing implicit instance method". A default body could therefore only use its parameters and statics, not the contract it belongs to. Look the method up in the interface as well and dispatch it through the vtable the object carries, so the call lands on the concrete implementation and a class override still wins over the default it replaces. Ordinary functions use their source name as the link name, so an Absolute function named after a C runtime entry point took over that symbol. The generated code and the Absolute runtime both call into those, so allocation started running user code: a program defining free(C*) printed the right answer and then aborted in managed cleanup, and one defining malloc corrupted itself before reaching main. Give such a definition a link name outside the C namespace. Parameter types alone are not enough because a zero-argument abort() still mangles to a bare abort, so the private name carries a leading dot, which no C identifier can have. extern, export "C", and main keep owning their symbols deliberately and are untouched, and every other function keeps the name it had. Both cases are covered by tests that fail without these changes: the interface build fails outright, and base-language segfaults. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
…ct arrays ConsumeTemplateClose split the `>>` that closes a nested argument list by inserting a token into `tokens`. That vector backs every Token* the parser holds, including ones live across the call, so the insert reallocated it and left them dangling: Pair<Pair<int32>> crashed the compiler with a segfault or bad_alloc and no diagnostic, while the same code with `> >` compiled. Split the token in place instead, leaving the vector's size alone: the call takes the first '>' and the token keeps the second for the enclosing list. Lambda captures named a parameter's symbol, but the analyzer never recorded an ExpressionInfo for the parameter declaration, so code generation mapped it to InvalidSymbolId and no capture could resolve to its storage. A lambda could close over a local but not over the enclosing function's parameters. Record the declaration when the parameter symbol is declared. Integer division by zero is undefined behavior in LLVM, so an unchecked sdiv let the program keep running on whatever the optimizer produced: the same expression printed a different garbage value on each run and exited successfully. Reject a divisor that is provably zero at compile time, and check the rest at runtime the way an array index is checked. Measured on the algorithm suite, the check costs nothing: gcd, the division-heavy case, is unchanged. A sized declarator was only recognized after a primitive type name, so `Point pts[3]` parsed as an inline value declaration and gave the variable the element type; indexing it then reported that a Point is not an array. Treat a sized declarator on the name as an array declaration for a user-defined element type too. Heap arrays of the same type already worked. Each fix has a test that fails without it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
HashMap and HashSet took a remainder to find a bucket and another one for every probe step, so each lookup paid several integer divisions on its hottest loop. The capacity is already a power of two everywhere it grows, so the remainder is a mask: keep capacity - 1 in a field, round a requested capacity up to a power of two so that stays true, and mask instead. The mask also bounds every probe index, so the probe reads no longer need a second bounds check, and it removes the sign fixup a remainder needed for a negative hash. Binding the comparison callable inside the probe loop cost a closure retain and release per step, because a callable field cannot be invoked directly and has to be copied to a local first. Bind it once per lookup instead. put probed twice for every new key, once to check for an existing entry and again for the insertion slot. One probe answers both: it returns early on a match and otherwise remembers the first tombstone. Growing still reprobes, since rehashing invalidates the slot. Vector's push checked the bounds of a store it had just proven in range, and its growth loop checked both buffers on every element. Both use the unchecked accessors the indexer already uses. The indexer's own bounds assert folds two signed compares into one unsigned compare. Measured with process CPU time, medians, on the collection benchmark: hashmap-insert-lookup 1.24x faster, vector-push-sum 1.20x faster, vector-sort unchanged. Every published checksum still matches. Keeping the growth loop inside push matters: moving it to its own method made push small enough to inline into the caller, which cost about 25% on the insertion-sort workload. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
…ler races TODO P0 asks for Linux Debug and Release to pass. Both now run 500/500. lli JITs a module by itself and links nothing, so every test that touched a managed pointer died resolving `absolute_managed_*`. The static archive cannot fill that gap: its thread-local accesses carry R_X86_64_GOTTPOFF relocations the JIT linker rejects. Build the runtime as a shared library where those tests exist and hand it to lli with --dlopen. That unmasked a real defect the link failure had been hiding, in a test whose own commit message claimed a full pass: MemoryStream.getBuffer returned `buffer.slice()`, and slice starts at the current position, so after writing it returned the untouched remainder rather than the bytes written, and the stream had no way to switch from writing to reading. getBuffer now returns what was written and MemoryStream has flip(). wasm-ld ships as a symlink to the one lld binary, which picks its flavor from argv[0]. Anything that resolves the symlink — as the corpus runner does — reaches it as plain `lld`, which refuses to link as a generic driver. Name the flavor explicitly when the linker resolves to lld. Two scheduler tests asserted the outcome of a race: * runtime-scheduler-metrics sampled the suspension window and the `completed` counter once each. Workers publish those, and a loaded machine gives no guarantee this thread runs inside any given window, so it now waits for the expected state with a timeout. * runtime-scheduler-stress required every leaf of a nested group to observe cancellation. Cancellation is cooperative and reaches only the children a group holds when it is cancelled, while a parent adds its children before checking its own flag, so a leaf that already ran cannot see it. The structural guarantee — every task runs and is joined — stays; the deterministic semantics are covered by runCancelVsComplete. Both used a `require` that called abort() with no output, which is why these failures arrived as empty logs. It now prints the line and the text of the failing check. Verified: 160/160 parallel stress runs, five consecutive full CTest runs at 500/500, and a separate Debug configuration at 500/500. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Closes the two open items under TODO P2. A metamorphic test needs no oracle for the value a program computes, only two programs that must agree. metamorphic_differential.py emits each case in six shapes a correct compiler has to treat identically: baseline, every declared name substituted, an uncalled function plus a branch guarded by a condition false for every reachable state, permuted top-level declarations, literals replaced by values the program computes, and all of those at once so a transformation mishandled only in combination still shows up. The shapes are generated from one template rather than rewritten textually, so a transformation cannot quietly change meaning. All of them must produce the same stdout and exit code. On a mismatch the runner shrinks the case while the disagreement survives and writes every shape's source, command and output. The optimization runner reported only the level that tripped, leaving out the outputs it disagreed with, which are what identifies the divergence; it now records the full source text and each level that already ran. Both reporting paths were checked by injecting a fault: making the dead branch reachable is caught as a divergence, minimized from 53 iterations to 1, with every shape recorded; breaking the oracle comparison produces a report carrying the source and the completed levels. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Covers the harness, corpus, minimization and budget items under TODO P3. Two in-process targets, for the lexer and for the lexer plus parser, built only where Clang's libFuzzer runtime is available so every other toolchain is unaffected. run_fuzzer.py replays the regression corpus first, fuzzes for a budget that ABSOLUTE_FUZZ_SECONDS widens for nightly and weekly runs, and on a crash minimizes the input and prints the command that reproduces it. Two corpora are kept: seeds give the fuzzer real language shapes to mutate, regressions hold inputs that once crashed. The parser was not ready to be fuzzed, and fixing that is most of this change: * It ended the process with std::exit on malformed input in 27 places, so a syntax error killed the caller. Every one now throws like the rest of Consume, which the CLI already reports and exits non-zero on. An embedder such as an LSP could not have used the parser before this. * Recursive descent turns source nesting into stack nesting, and about 25000 levels segfaulted. A depth guard reports an ordinary syntax error instead; 3000-term expressions and 200-deep parentheses still compile, so real code is unaffected. Memory and time are bounded by the runner. The parser fuzzer then found two null dereferences within seconds, both where a token was used after the source had ended: the uninitialized variable declaration path, and the initializer check in an instance declaration, minimized by the runner to the three bytes "S a". Both are fixed and both inputs are in the regression corpus. After the fixes a four-way fork run of 35000 executions reports no crashes, timeouts or OOMs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Continues TODO P3. Raw bytes reach the analyzer only by accident: almost every random buffer fails to parse, so a byte-oriented target would exercise the parser's error path instead. This harness spends the fuzzer's input as decisions in a generator that emits only syntactically valid programs, so what arrives at the analyzer is a real AST whose shape the fuzzer controls. Semantic validity is deliberately not a goal. Type mismatches, bad calls and odd conversions are what the analyzer exists to report, and reporting them has to be a diagnostic rather than a crash. The difference shows in coverage: 1978 edges against 82 for the parser target on a comparable budget. No crashes so far in a four-way fork run. Seeds are now per target, because the analyzer consumes decision streams rather than source. Since such a buffer is not readable on its own, ABSOLUTE_FUZZ_DUMP_SOURCE prints the program it stands for, which is what a crash report actually needs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Closes the coverage item under TODO P3. The edge count libFuzzer prints says how much instrumented code a corpus reached, not where the gaps are, so a target can sit at a healthy number while never entering whole areas of the front end. fuzz_coverage.py replays a corpus through an instrumented target and splits the result by component, because those fail independently: a corpus can saturate the lexer and never reach semantic analysis. The instrumentation has to cover the libraries the harness links, not just the harness, or the report describes one translation unit; the first attempt did exactly that and reported 210 regions with no source file in it. The flags are therefore applied to the whole build, behind an option that is off by default because it slows everything down. The first run turned into two concrete tasks: * The analyzer target reaches 20.7% of the analyzer and 32.4% of the parser, against 57.5% for the parser target, so its generator emits too narrow a language: no classes, interfaces, generics, pointers, arrays or exceptions. * The dedicated lexer target covers 30.0% of the lexer while the parser target covers 65.6% of it on the same corpus, so that harness is reaching less of its own subject than a target that only uses it in passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
The coverage build directory was staged before .gitignore covered it. Only tools/testing/fuzz_coverage.py, the CMake option and the TODO note belong in that change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Four independent failures kept "CI required gate" red: - OptimizationLevel was only declared in codegen.h, which pch.h includes under ABSOLUTE_HAS_LLVM. Absolute-Compiler.cpp uses it unconditionally, so every ABSOLUTE_ENABLE_LLVM=OFF build failed to compile the driver (build-and-test on windows-2022 Debug/Release and termux-host-contract). Move the enum to its own LLVM-free header, include it unconditionally, and put Absolute-CodeGen/include on the driver's include path so the header resolves without the CodeGen target. - llvm-compatibility ran in a bare ubuntu:24.04 container without zlib1g-dev. LLVM's exported LLVMSupport target lists ZLIB::ZLIB in its link interface, so find_package(LLVM) aborted at generate time for LLVM 18, 19 and 20. Install zlib1g-dev alongside libzstd-dev. - macos-smoke could not build Absolute-Runtime: Darwin's <ucontext.h> errors out unless _XOPEN_SOURCE is defined, leaving getcontext, makecontext and swapcontext undeclared. Define it for the Apple branch of scheduler_fiber.h. - windows-llvm-release timed out bootstrapping the ~1 GB LLVM SDK on a cache miss, and the always() diagnostics step then threw on the missing llvm-config.exe, failing the job at that step and hiding the timeout. Guard the lookup and give the bootstrap enough headroom. Verified locally by building the driver both with ABSOLUTE_ENABLE_LLVM=OFF and ON, and by reproducing the LLVM export failure through a missing zstd/zlib target. The Darwin and Windows paths are preprocessor- and workflow-only changes that cannot be compiled here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Fixing the zlib gap in llvm-compatibility let those jobs get past CMake and reach the compiler, which exposed two API guards that never matched the versions the project claims to support (18 through 21). - LazyValueInfo::getPredicateAt returns Constant* rather than the removed Tristate enum since LLVM 19, but the guard in codegen.cpp opened at 21. LLVM 19 and 20 therefore took the old branch and failed with "no member named 'True' in 'llvm::LazyValueInfo'". - LLVM 20 added a ThinOrFullLTOPhase argument to the optimizer-last extension point callback, so the sanitizer pass registration in codegen_module.cpp no longer converted to the expected std::function. Split the registration per version and keep the pass setup shared. The remaining >= 21 guards were checked and are genuinely LLVM 21 API changes (CaptureInfo replacing NoCapture, Triple-based TargetMachine entry points), so they are left alone. Verified by installing LLVM 18, 19 and 20 locally and building Absolute-CodeGen and the absolutec driver against each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
The LLVM and WASI bootstrap steps captured their output into a variable and only wrote it to the log after the command returned. When the LLVM step hit its timeout it produced no output at all for the full 25 minutes, so the job log showed only the timeout with no indication of how far the download had progressed. Pipe through Tee-Object instead, which writes as output arrives. This is a diagnostics change only. It does not address the underlying problem that a cold cache requires downloading an ~1 GB SDK inside the step timeout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
The previous commit adapted the sanitizer callback at its call site, which was the wrong layer and broke LLVM 21: codegen_pch.h already ships an OptimizerLastEPCallback shim plus a macro that rewrites the registration, so call sites are meant to keep passing the two-argument form and let the shim add the LTO phase. Passing three arguments made the shim call the callback with two. The real defect is the shim's guard. LLVM 20 added the LTO-phase argument, but the shim only activated at 21, so LLVM 20 had no adapter at all. That guard also covered the unrelated Triple-based target APIs, which really are LLVM 21, and bundling them hid the mismatch. Split the block: the optimizer callback shim now activates at 20, the Triple helpers stay at 21, and codegen_module.cpp goes back to the plain two-argument lambda. LLVM 21 therefore returns to exactly the configuration that was passing before, now shared with LLVM 20. Verified by building absolutec against LLVM 18, 19 and 20 locally. LLVM 21 is not installable in this environment; it is covered by the restored call site being byte-identical to the previously passing one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
With the build failures fixed, these jobs reached their test phase for the first time and exposed two test-side assumptions. Four debug-info tests emit IR, link through --build-exe and inspect the result with the LLVM tools, but they were registered unconditionally. In any ABSOLUTE_ENABLE_LLVM=OFF configuration they failed with "LLVM backend is unavailable in this build", which is the configuration working as intended rather than a defect. Guard them with ABSOLUTE_ENABLE_LLVM, the pattern already used elsewhere in this file; the neighbouring semantic and diagnostic debug-info tests are frontend only and stay unguarded. This covers termux-host-contract and the windows-2022 build-and-test jobs, all of which configure the backend off. The debug-info IR assertion also required the llvm.dbg.declare intrinsic spelling. LLVM 19 prints debug info as records instead, so the check fails from 19 onward even though the emitted debug info is correct. Accept either spelling. Verified locally: the termux contract configuration goes from 4 failures to 178/178, the debug-info tests pass against both LLVM 18 (24 intrinsics) and LLVM 19 (23 records), and the full suite is 501/501 on LLVM 18. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
The earlier fix defined _XOPEN_SOURCE inside scheduler_fiber.h, which tasks.cpp includes after about twenty standard library headers. That is enough to satisfy the #error in Darwin's <ucontext.h> and make the build succeed, but a feature macro only affects headers included after it: the Darwin headers libc++ had already pulled in were parsed without it. The declarations and the ucontext layout can therefore disagree within one translation unit, which is consistent with the Bus error and SIGTRAP the scheduler and async tests now hit on macOS. Set the macro from the command line for that source file instead, so it applies before any header, and pair it with _DARWIN_C_SOURCE so the stricter feature set does not hide BSD extensions from the rest of the unit. The header now states the requirement and fails loudly if it is missing, rather than silently defining it too late. This removes a real class of undefined behaviour. It is not confirmed to be the cause of the macOS failures: if they persist, the remaining explanation is that the ucontext routines are unusable on arm64 Darwin, which needs a different fiber backend and is a design decision rather than a fix. Verified on Linux, where the Apple branch is inert: tasks.cpp compiles and Absolute-Runtime links. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
The streamed bootstrap output shows the ~1 GB SDK download finishing in 16 seconds at ~56 MB/s, after which the step spends the remaining 24.5 minutes inside tar.exe unpacking the .tar.xz before being killed. The comment added with the timeout increase claimed the step was killed mid-download, which is wrong and would point a reader at the wrong phase. Comment only; the 25-minute value is left as is. Raising it further would not help, since the overrun is in extraction rather than transfer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
The streamed bootstrap output showed where windows-llvm-release actually spends its time: the ~1 GB SDK downloads in 16 seconds at ~56 MB/s, then tar.exe runs for the remaining 24.5 minutes and is killed. Real-time antivirus scanning of every file written is the usual cause of that ratio on CI runners, since the archive expands to several GB of mostly small headers. Add a best-effort Defender exclusion for the toolchains directory before extraction, and report how long extraction took. The exclusion is wrapped in a try/catch so a machine without Defender, or without the privileges to configure it, still extracts as before. The timing line makes the next run informative either way: if extraction is still slow, the number rules the antivirus theory out rather than leaving it open. Not verifiable from this environment, which has no Windows runner and no PowerShell to parse the script; CI is the test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
The instrumentation added in the previous commit settled where the time goes. The download finishes in 9 seconds at ~110 MB/s, the Defender exclusion is applied successfully, and tar.exe still consumes the entire remaining 24.8 minutes without returning. So neither transfer nor antivirus scanning explains it: bsdtar decoding this xz stream on a single thread does. 7-Zip ships on the hosted Windows images and decodes xz in parallel. Split extraction into an xz pass and a tar pass through it, timing each, and keep tar.exe as the fallback for any machine where 7-Zip is missing or the pass does not produce the expected archive. The Defender exclusion is retained: it was not the fix, but it is cheap and large file writes are genuinely slower under real-time scanning. The per-phase timings mean the next run distinguishes slow decompression from slow file creation, whichever way this lands. Not verifiable here: no Windows runner and no PowerShell to parse the script, so CI is the test. The fallback keeps the failure mode no worse than the current one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Building the stress programs with --sanitize=address surfaced a leak that reproduces on every run of collection-snapshot-stress: 6048 bytes in 12 allocations, attributed to std.collections.Map.put and Set.add. The growth paths allocate a larger array, copy the live elements across, and then assign it with copy(...). That stores a duplicate in the field and leaves the freshly allocated local unreferenced and unreleased. The arithmetic matches exactly: six doublings from capacity 4 to 256 leak 8*(8+16+32+64+128+256) = 4032 bytes for the 8-byte KeyValuePair entries and 4*(...) = 2016 for the 4-byte set items, 6048 together, in 6 objects each. Assign with move(...) so the field takes ownership of the array that was just built. vector.abs already does exactly this at its own growth sites, so this brings map, set, deque and priority_queue in line with the idiom the standard library had already settled on rather than introducing a new one. A plain assignment is rejected by the analyzer, which names move(...) among the accepted forms. The remaining copy(...) uses are left alone: iterator snapshots, the copy-on-write in ensureUnshared, and the rehash locals in hash_map are all genuine copies. Verified with ASan: collection-snapshot-stress goes from 6048 bytes leaked to a clean exit. The full suite is 501/501 on LLVM 18. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SPqntJmyXNSmZNzuQWj5QS
Fixes ten defects that kept the CI required gate red: OptimizationLevel unavailable in LLVM-free builds, missing zlib1g-dev in the llvm-compatibility container, Darwin _XOPEN_SOURCE for the ucontext fiber routines, LazyValueInfo and optimizer-callback version guards that never matched the supported LLVM 18-21 range, four debug-info tests registered without a backend guard, an LLVM-18-only IR assertion, a diagnostics step that failed its own job and masked the real cause, bootstrap output discarded on timeout, bsdtar extraction of the Windows SDK replaced with 7-Zip, and a copy-on-grow ownership bug in the std collections. That last one took macos-smoke from 17 failures to 1. Remaining and tracked separately: absolute.run-sanitizer-ownership-stability on macOS, and an intermittent Linux failure that reproduces only in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.