Skip to content

WASM agent coverage#88

Merged
jayvdb merged 3 commits into
mainfrom
wasm-agent-coverage
Jul 18, 2026
Merged

WASM agent coverage#88
jayvdb merged 3 commits into
mainfrom
wasm-agent-coverage

Conversation

@jayvdb

@jayvdb jayvdb commented Jul 18, 2026

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • New Features

    • Added browser-based WebAssembly client tests covering connection states, offline queuing, reconnection, callbacks, DOM helpers, and live WebSocket interactions.
    • Added ChromeDriver setup for browser testing across supported platforms.
    • Added a dedicated coverage workflow for browser WebAssembly components.
  • Bug Fixes

    • Improved coverage reporting by incorporating browser-agent and server coverage data.
  • Refactor

    • Updated WebSocket test server startup to support caller-selected ports and clean shutdown.

@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The PR adds browser-based ws-wasm-agent coverage tests, a coverage WebSocket server, ChromeDriver mise tooling, and a wasm-agent-cov task that merges wasm and native coverage into lcov.info.

Changes

WebAssembly coverage integration

Layer / File(s) Summary
ChromeDriver tooling and policy
.mise/config.toml, config/conftest/policy/mise/mise.rego
Adds a pinned OS-scoped ChromeDriver tool, updates the Chrome browser task to resolve it with mise which, and allowlists the tool.
Coverage WebSocket server lifecycle
services/ws-test-server/Cargo.toml, services/ws-test-server/src/lib.rs, services/ws-test-server/src/bin/cov-server.rs, .codacy.yaml
Adds the coverage server binary, readiness-marker shutdown flow, explicit-port startup support, its filesystem dependency, and a Codacy exclusion for the trusted marker path.
Browser client coverage tests
services/ws-wasm-agent/Cargo.toml, services/ws-wasm-agent/tests/client.rs
Adds the coverage feature and browser tests for client configuration, connection behavior, queuing, callbacks, DOM helpers, JavaScript reflection, live messaging, reconnection, and tracing.
Coverage collection and aggregation
.mise/config.coverage.toml, .github/workflows/coverage.yaml
Adds the wasm-agent-cov task and invokes it from CI to process wasm and native profiles into lcov.info.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Workflow as GitHub Actions coverage
  participant Mise as wasm-agent-cov
  participant Server as cov-server
  participant Tests as wasm-bindgen tests
  participant LLVM as LLVM coverage tools
  participant Lcov as lcov.info
  Workflow->>Mise: Run coverage task with CHROMEDRIVER
  Mise->>Server: Start instrumented server
  Mise->>Tests: Run browser tests
  Tests-->>Mise: Emit wasm coverage profiles
  Mise->>LLVM: Merge profiles and export lcov
  Server-->>LLVM: Provide native coverage
  LLVM->>Lcov: Append filtered coverage
Loading

Suggested reviewers: pierre-tenedero

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main change: adding coverage support for the WASM agent and related test tooling.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch wasm-agent-coverage

Comment @coderabbitai help to get the list of available commands.

@deepsource-io

deepsource-io Bot commented Jul 18, 2026

Copy link
Copy Markdown

DeepSource Code Review

We reviewed changes in 37f9429...7662d6a on this pull request. Below is the summary for the review, and you can see the individual issues we found as inline review comments.

See full review on DeepSource ↗

PR Report Card

Overall Grade  

Focus Area: Reliability
Security  

Reliability  

Complexity  

Hygiene  

Code Review Summary

Analyzer Status Updated (UTC) Details
C# Jul 18, 2026 4:21a.m. Review ↗
C & C++ Jul 18, 2026 4:21a.m. Review ↗
Docker Jul 18, 2026 4:21a.m. Review ↗
Java Jul 18, 2026 4:21a.m. Review ↗
JavaScript Jul 18, 2026 4:21a.m. Review ↗
Python Jul 18, 2026 4:21a.m. Review ↗
Rust Jul 18, 2026 4:21a.m. Review ↗
Secrets Jul 18, 2026 4:52a.m. Review ↗
Code coverage Jul 18, 2026 4:52a.m. Review ↗

@codacy-production

codacy-production Bot commented Jul 18, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 20 complexity · 0 duplication

Metric Results
Complexity 20
Duplication 0

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

Run reviewer

TIP This summary will be updated as you push new changes.

@codecov

codecov Bot commented Jul 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@jayvdb
jayvdb marked this pull request as ready for review July 18, 2026 04:21
@jayvdb

jayvdb commented Jul 18, 2026

Copy link
Copy Markdown
Member Author

@deepsourcebot review

@codacy-production codacy-production Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR establishes a robust coverage infrastructure for the WASM agent using headless Chrome and a specialized test server (cov-server). All core acceptance criteria and recommended test scenarios have been implemented. However, the PR description is currently empty, providing no context for these infrastructure changes.

While the code is considered 'Up to Standards' by Codacy, the new integration test file services/ws-wasm-agent/tests/client.rs has high complexity (delta +20) and contains several reliability issues. Specifically, the suite lacks proper environment isolation due to DOM pollution and non-idempotent tracing initialization, which can cause intermittent failures or panics in CI. These issues should be resolved to ensure a stable testing pipeline.

About this PR

  • The PR description is empty. Please provide a high-level summary of the infrastructure changes, specifically how the headless Chrome environment and the cov-server interact to produce coverage results.

Test suggestions

  • Verify agent state transitions from disconnected to connecting and finally connected.
  • Verify offline message queuing behavior and the dropping of oldest messages when capacity (1000) is exceeded.
  • Verify reconnection logic and exponential backoff when a connection attempt fails.
  • Verify DOM helper functions for setting and appending text to textarea elements.
  • Verify JS reflection helpers for reading numeric, boolean, and nested object fields.
  • Verify full end-to-end connectivity and message dispatch against the instrumented cov-server backend.

TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback

let target = document.create_element("textarea").unwrap();
target.set_id("ta-target");
let _appended = body.append_child(&target).unwrap();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

Appending elements to the document body without removing them causes DOM pollution. Tests should be isolated; consider removing the elements at the end of the test.

Try running the following prompt in your coding agent:

Refactor the textarea_helpers_set_and_append test in services/ws-wasm-agent/tests/client.rs to ensure that all elements appended to the document body are removed at the end of the test, ideally using an RAII guard or explicit remove_child calls.

fn tracing_initializes() {
// tracing_wasm's global default can only be installed once per wasm instance, so this is the only caller.
init_tracing();
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

Suggestion: Tracing initialization can only happen once per instance. To prevent the test suite from panicking if this test is executed multiple times or in parallel, the underlying initialization should be made idempotent.

Try running the following prompt in your coding agent:

Modify the init_tracing function in et_ws_wasm_agent to use std::sync::Once so it can be safely called multiple times without panicking.


#[wasm_bindgen_test]
fn config_setters_and_initial_state() {
let mut config = WsClientConfig::new("ws://127.0.0.1:8080/ws".to_string());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ LOW RISK

Nitpick: The SERVER_URL constant is defined for centralizing the test server address, but this location still uses a hardcoded string. Use the constant to ensure consistency.

Suggested change
let mut config = WsClientConfig::new("ws://127.0.0.1:8080/ws".to_string());
let mut config = WsClientConfig::new(SERVER_URL.to_string());

@@ -0,0 +1,266 @@
//! Server-independent coverage of the WASM agent's client logic.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ LOW RISK

Nitpick: The documentation states this file exercises logic 'without a server', but connects_flushes_queue_and_sends depends on a live backend. Update the header to clarify that while most tests are offline, some integration coverage requires the test server.

This might be a simple fix:

Suggested change
//! Server-independent coverage of the WASM agent's client logic.
//! Coverage of the WASM agent's client logic.

Comment on lines +54 to +55
let storage = web_sys::window().unwrap().local_storage().unwrap().unwrap();
storage.clear().unwrap();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`storage.clear()` erases unrelated `localStorage` keys used by other tests


storage.clear() mutates shared browser state for the whole origin. When tests execute in different orders, this can break unrelated cases and create flaky CI failures that are hard to reproduce.

Remove broad clearing and avoid asserting globally persisted state here. Keep the test focused on WsClientConfig setters and disconnected state.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
.mise/config.toml (1)

242-260: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Add per-platform checksums for the ChromeDriver archives. Versioned URLs alone don’t pin the archive contents; use per-platform checksum fields or a mise.lock entry.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.mise/config.toml around lines 242 - 260, Add integrity verification for the
archives configured under tools."http:chromedriver" by supplying the correct
per-platform checksum for each linux-x64, macos-arm64, and macos-x64 URL. Keep
the existing version, platform selection, URLs, and binary configuration
unchanged; use checksum fields or the repository’s established mise.lock
mechanism.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.mise/config.coverage.toml:
- Line 208: Update the EXIT trap in the coverage configuration to remove the
readiness marker and reap the instrumented server during shutdown, ensuring this
cleanup runs even when setup fails before main returns. Retain kill only as a
bounded fallback after the normal shutdown/reaping path, while preserving the
existing failure cleanup behavior.

In `@services/ws-wasm-agent/tests/client.rs`:
- Line 8: Replace the crate-level target_arch gate in the client test target so
non-wasm32 invocations fail loudly instead of silently running zero tests.
Direct the failure message to the wasm32 mise task, while preserving normal
browser test execution on wasm32.

---

Nitpick comments:
In @.mise/config.toml:
- Around line 242-260: Add integrity verification for the archives configured
under tools."http:chromedriver" by supplying the correct per-platform checksum
for each linux-x64, macos-arm64, and macos-x64 URL. Keep the existing version,
platform selection, URLs, and binary configuration unchanged; use checksum
fields or the repository’s established mise.lock mechanism.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 910bfb83-0689-4ab1-9cf8-3d136d59be9f

📥 Commits

Reviewing files that changed from the base of the PR and between 37f9429 and 7662d6a.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (10)
  • .codacy.yaml
  • .github/workflows/coverage.yaml
  • .mise/config.coverage.toml
  • .mise/config.toml
  • config/conftest/policy/mise/mise.rego
  • services/ws-test-server/Cargo.toml
  • services/ws-test-server/src/bin/cov-server.rs
  • services/ws-test-server/src/lib.rs
  • services/ws-wasm-agent/Cargo.toml
  • services/ws-wasm-agent/tests/client.rs

RUSTFLAGS="-Cinstrument-coverage" cargo build -q -p et-ws-test-server --bin cov-server
LLVM_PROFILE_FILE="{{ config_root }}/$covdir/cov-server-%p.profraw" ./target/debug/cov-server "$covdir/server-ready" &
server_pid=$!
trap 'kill "$server_pid" 2>/dev/null || true' EXIT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Preserve the clean shutdown protocol on failure.

The EXIT trap directly kills the instrumented server, so failures before Line 224 prevent main() from returning and
can discard its native profile. Have the trap remove the readiness marker and reap the process, with kill only as a
bounded fallback.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.mise/config.coverage.toml at line 208, Update the EXIT trap in the coverage
configuration to remove the readiness marker and reap the instrumented server
during shutdown, ensuring this cleanup runs even when setup fails before main
returns. Retain kill only as a bounded fallback after the normal
shutdown/reaping path, while preserving the existing failure cleanup behavior.

//! textarea helpers, and the JS reflection helpers -- so it runs headless with no backend. It is what the
//! `wasm-agent-cov` mise task builds instrumented to measure the agent's coverage.
#![cfg(test)]
#![cfg(target_arch = "wasm32")]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Do not silently disable the entire test target outside wasm32.

This makes an incorrectly invoked test command succeed with zero browser tests. Obtain an explicit exception or replace
the silent gate with a fail-loud guard directing callers to the wasm32 mise task.

As per coding guidelines, tests must never be platform-disabled without explicit user approval, and missing tools must
fail loudly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@services/ws-wasm-agent/tests/client.rs` at line 8, Replace the crate-level
target_arch gate in the client test target so non-wasm32 invocations fail loudly
instead of silently running zero tests. Direct the failure message to the wasm32
mise task, while preserving normal browser test execution on wasm32.

Source: Coding guidelines

@jayvdb
jayvdb requested a review from pierre-tenedero July 18, 2026 04:44
@jayvdb
jayvdb merged commit a919903 into main Jul 18, 2026
32 of 36 checks passed
@jayvdb
jayvdb deleted the wasm-agent-coverage branch July 18, 2026 10:01
This was referenced Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants