[issues/510] Add manual QA environments (Ubuntu Docker, Cursor) and BATS tests for QA scripts#602
Merged
Conversation
…ATS tests for QA scripts ## Summary Adds discoverable manual QA environments for platform-specific and IDE-specific test cases. Ubuntu TCs now have a Docker-based desktop environment with VS Code pre-configured. Cursor TCs get a guide file with copy-pasteable commands. Adds 808 lines of BATS tests achieving full branch coverage on resolve-qa-labels.js, the QA label resolution engine used by all QA scripts. ## Changes - Ubuntu Docker QA: Dockerfile (Ubuntu 24.04 + XFCE + VS Code), entrypoint that generates TC checklist, builds extension, and launches noVNC desktop at localhost:6080 - Cursor QA: guide file written to fixture workspace root with TC IDs, scenarios, and commands for assisted TCs - npm scripts: test:release:ubuntu, test:release:cursor for discoverable access to both environments - BATS tests: 129 tests (808 lines) covering all branches of resolve-qa-labels.js — argument parsing, YAML auto-discovery, parsing, JSON grouping, filtering, and output formats - .vscodeignore cleanup: excludes qa/, test-fixtures/, dev docs, temp files, and Docker artifacts from .vsix (was shipping 4MB+ of test data) - Docker volume for node_modules: isolates Linux-native binaries from host macOS install ## Test Plan - [ ] All 1979 unit tests pass - [ ] All 129 BATS tests pass - [ ] Manual: `pnpm test:release:ubuntu` builds and launches Ubuntu desktop with extension installed - [ ] Manual: `pnpm test:release:cursor` prints guide and launches Cursor with fixture workspace - [ ] Manual: `pnpm test:release:grep <id>` works inside Ubuntu container ## Related - Closes #510
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
WalkthroughAdds a QA schema field ChangesQA Automation: Assisted Conversion & Non-Automatable Reasons
Sequence Diagram(s)(omitted) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
New manual QA environments for Ubuntu and Cursor IDE-specific tests were introduced, requiring additional test cases. Suggested test cases:
Generated by QA Gap Check (GPT-4o-mini via GitHub Models) |
couimet
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/rangelink-vscode-extension/docker/entrypoint.sh`:
- Line 70: The extension install command currently swallows failures via "||
true" so the container can report ready even if the VSIX failed to install;
change the entrypoint logic around the "DISPLAY=:1 code --install-extension
\"$VSIX\"" invocation to let failures surface: run the install command without
"|| true", capture its exit status, log a descriptive error including "$VSIX"
and the install stderr, and exit non‑zero (e.g., exit 1) when the install fails
so the container fails fast and CI/QA see the problem.
In `@packages/rangelink-vscode-extension/scripts/qa-ubuntu-docker.sh`:
- Line 75: The Docker run/publish in the QA script currently exposes noVNC to
all interfaces via the "-p 6080:6080" mapping; change that mapping to bind to
loopback only (e.g., use "127.0.0.1:6080:6080" or the equivalent "--publish
127.0.0.1:6080:6080") in the script line containing "-p 6080:6080" so the VNC
web UI is only accessible from localhost.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 79a7b9d2-9bdf-483f-856f-c1d758f6af5f
📒 Files selected for processing (16)
.gitignorepackage.jsonpackages/rangelink-vscode-extension/.vscodeignorepackages/rangelink-vscode-extension/TESTING.mdpackages/rangelink-vscode-extension/docker/Dockerfile.ubuntupackages/rangelink-vscode-extension/docker/entrypoint.shpackages/rangelink-vscode-extension/package.jsonpackages/rangelink-vscode-extension/qa/qa-test-cases-v1.1.0.yamlpackages/rangelink-vscode-extension/scripts/generate-qa-issue.shpackages/rangelink-vscode-extension/scripts/generate-qa-test-plan.shpackages/rangelink-vscode-extension/scripts/generate-release-testing-instructions.shpackages/rangelink-vscode-extension/scripts/qa-cursor.shpackages/rangelink-vscode-extension/scripts/qa-ubuntu-docker.shpackages/rangelink-vscode-extension/scripts/resolve-qa-labels.jspackages/rangelink-vscode-extension/src/__integration-tests__/suite/builtInAiAssistants.test.tstests/shell/resolve-qa-labels.bats
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…ilure detection Bind noVNC port to 127.0.0.1 so the auth-less VNC desktop is not exposed to the local network. Surface extension install failures in the entrypoint instead of swallowing them with || true. Ref: #602 (review)
✅ CI / Integration Tests (with extensions) — run summary
|
✅ CI / Test & Validate — run summary
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds discoverable manual QA environments for platform-specific and IDE-specific test cases. Ubuntu TCs now have a Docker-based desktop environment with VS Code pre-configured. Cursor TCs get a guide file with copy-pasteable commands. Adds 808 lines of BATS tests achieving full branch coverage on resolve-qa-labels.js, the QA label resolution engine used by all QA scripts.
Changes
Test Plan
pnpm test:release:ubuntubuilds and launches Ubuntu desktop with extension installedpnpm test:release:cursorprints guide and launches Cursor with fixture workspacepnpm test:release:grep <id>works inside Ubuntu containerRelated
Summary by CodeRabbit
Tests
Documentation
Chores