Skip to content

On Device Test Plan

Jump to bottom
Michael DeGuzis edited this page Apr 9, 2026 · 4 revisions

On-Device Test Plan

Manual test checklist for validating Decky Proton Pulse on real Steam Deck hardware. Run this after every significant change before considering a build release-ready.

See Developer Guide for how to deploy to your Deck and enable CEF debugging.

Contents

Prerequisites

  • Decky Loader installed on the Steam Deck
  • Plugin deployed via bash scripts/deploy.sh --target stable --deck-ip <IP>
  • CEF remote debugging enabled (Decky Settings -> Allow Remote CEF Debugging)
  • At least one game in your library with ProtonDB reports (e.g. Elden Ring, app ID 1245620)
  • SSH access to the Deck for log inspection
  • Note whether the Steam client is on stable or beta before starting the run

1. Deployment Verification

  • Plugin appears in the Decky sidebar (lightning bolt icon visible)
  • Record Steam client channel: stable / beta
  • No errors in plugin loader journal: 
    ssh deck@192.168.1.x "journalctl -u plugin_loader -n 50 --no-pager | grep -i 'proton pulse\|error\|exception'"
  • Proton Pulse initializing appears in the journal output
  • Log file created at /tmp/decky-proton-pulse.log: 
    ssh deck@192.168.1.x "cat /tmp/decky-proton-pulse.log"

2. System Detection

Open the Decky sidebar -> Proton Pulse panel.

  • No crash on panel open
  • In CEF DevTools console (http://192.168.1.x:8081), call get_system_info manually or inspect the log for the detected values
  • Log shows detected CPU (should be AMD Ryzen 9 9950X3D or your actual CPU)
  • Log shows detected GPU (should be NVIDIA GeForce RTX 5080 or your actual GPU)
  • gpu_vendor is correctly set to nvidia / amd / intel / other
  • driver_version detected (not null)
  • proton_custom detected if custom Proton is installed in ~/.steam/root/compatibilitytools.d/
  • Any null fields are logged at WARN level (check /tmp/decky-proton-pulse.log)

3. Sidebar Panel

Navigate to a game page in the Steam library (no game running).

  • Sidebar shows the game name and badge preview once a game page is focused
  • "Check ProtonDB ▶" button is enabled when on a game page with no game running
  • "Check ProtonDB ▶" button description reads "Navigate to a game first" when no game is focused
  • Debug Logs toggle is present and responds to taps
  • Log viewer section is visible and shows log content (or "No logs yet." placeholder)
  • Log viewer auto-scrolls to the bottom

4. ProtonDB Badge

Navigate to a game page with ProtonDB reports.

  • Badge appears in the sidebar panel showing GPU tier + top rating (e.g. ⚡ PP·NVIDIA Platinum)
  • Badge shows ? No Data for a game with no ProtonDB reports
  • Badge is hidden (not shown as broken/errored) if the ProtonDB summary request fails
  • Badge position on game page: badge appears to the right of the existing ProtonDB badge on the game detail page
    • Confirm it does not replace or overlap the existing badge
    • If positioning is off, note the exact Steam DOM path using CEF DevTools and open an issue

5. Ranked Report Modal

Click "Check ProtonDB ▶" on a game with reports.

  • Loading toast appears: "Fetching ProtonDB reports..."
  • Modal opens with a list of ranked report cards
  • GPU filter tabs are present: the user's GPU tier tab is selected by default
    • NVIDIA tab shown and pre-selected for NVIDIA systems
    • AMD tab shown and pre-selected for AMD systems
    • "Other" tab shown and pre-selected for Intel / unknown GPU systems
  • Switching filter tabs updates the visible cards correctly
  • Report cards show: score badge, GPU tier badge, Proton version, recency, truncated notes
  • Score badge color reflects rating (platinum = highest, borked = lowest)
  • Cards are sorted by score descending within each GPU tier
  • Selecting a card via checkbox deselects any previously selected card (radio behavior)
  • "Exit" closes the modal with no changes
  • Modal shows "No reports found for this game." for a game with no reports (and no crash)
  • Manage This Game does not crash on Steam beta
  • Manage This Game does not crash on Steam stable
  • If beta and stable behave differently, capture both a screenshot and synced logs

6. Apply Launch Options

Warning: This writes to Steam launch options. Test with a non-critical game first. Check the launch options before and after.

Before testing: note the current launch options for the test game. Steam Library -> right-click game -> Properties -> General -> Launch Options.

  • Select a report card in the modal
  • Click "Apply ▶"
  • Success toast appears: "Launch options applied for [Game Name]"
  • Verify launch options were written:
    • Steam Library -> right-click game -> Properties -> General -> Launch Options
    • Should contain PROTON_VERSION="..." and %command%
  • The written Proton version matches the selected report's protonVersion field shown on the card
  • Click "Clear" in the modal
  • Verify launch options are now empty
  • "Exit" after Clear leaves launch options empty (no unintended re-apply)

7. Logging

  • Log file at /tmp/decky-proton-pulse.log is written during normal use
  • Log entries include timestamps and level prefix: [YYYY-MM-DD HH:MM:SS] [INFO] ...
  • Enable Debug Logs toggle in the sidebar - log viewer shows DEBUG-level entries
  • Disable Debug Logs toggle - DEBUG entries stop appearing
  • Log viewer in the sidebar reflects new log entries within ~3 seconds (polling interval)
  • Log viewer auto-scrolls to the bottom on new content
  • Log file does not grow unbounded - rotation kicks in at 5MB (hard to test manually; inspect RotatingFileHandler config in main.py)

8. Game-Running Guard

  • Launch a game
  • Open Decky sidebar -> Proton Pulse panel
  • "Check ProtonDB ▶" button is disabled with description "Quit your game first"
  • Button re-enables within ~5 seconds of quitting the game
  • Attempting to click Apply while a game is running does nothing (button should already be disabled before the modal is opened, but verify no race condition)

9. Error Handling

Network errors

  • Disconnect network, navigate to a game page -> badge hidden (no error shown to user)
  • Reconnect, navigate away and back -> badge reappears on next focus

No reports

  • Navigate to a game with no ProtonDB reports (obscure title or test with app ID 0)
  • Badge shows ? No Data
  • Clicking "Check ProtonDB ▶" shows toast: "No ProtonDB reports found for this game."
  • No crash or unhandled error

Steam beta / stable compatibility

  • Re-run the same game flow on Steam beta when available
  • Re-run the same game flow on Steam stable when available
  • Note whether any crash is reproducible only on one channel
  • If behavior differs, document which UI surface breaks:
    • Decky sidebar panel
    • Manage This Game
    • Logs tab
    • Settings / About

Apply failure

  • If SetAppLaunchOptions fails (hard to trigger manually - could test via CEF console: SteamClient.Apps.SetAppLaunchOptions with invalid args)
  • Toast shows "Failed to apply - check logs"
  • Error is logged to /tmp/decky-proton-pulse.log

10. ProtonDB API Field Verification

Verify that the live API response fields match the types defined in src/types.ts. Run from the Deck's terminal (or SSH in) to avoid rate limiting on a non-Deck IP.

Summary endpoint

curl -s "https://www.protondb.com/api/v1/reports/summaries/1245620.json" | python3 -m json.tool

Check each field against ProtonDBSummary in src/types.ts:

Field Expected type Actual value
score number (float)
tier string (ProtonRating)
total number
trendingTier string (ProtonRating)
bestReportedTier string (ProtonRating)
confidence string
  • All fields present and types match
  • If any field is missing or renamed, update src/types.ts and src/components/Badge.tsx

Reports endpoint

curl -s "https://www.protondb.com/api/v1/reports/app/1245620" | python3 -m json.tool | head -60

Check each field against ProtonDBReport in src/types.ts:

Field Expected type Actual value
timestamp number (Unix seconds)
rating string (ProtonRating)
protonVersion string
notes string
responses.gpu string (optional)
responses.gpuDriver string (optional)
responses.os string (optional)
responses.ram number (optional)
responses.kernel string (optional)
responses.cpu string (optional)
  • All fields present and types match
  • If any field is missing or renamed, update src/types.ts and src/lib/scoring.ts

11. Screenshot Gallery Flow Verification

Use this section when refreshing [[UI Screenshot Gallery]] on a real Deck.

Expected Gallery Order

  • Manage Game
  • Manage Configurations
  • Compatibility Tools
  • Logs
  • Settings
  • Issue Reporting
  • About

Verify Current Wired Shots

  • manage-game/default
  • manage-game/report-detail
  • manage-game/hardware-compare
  • manage-game/edit-modal
  • manage-configurations/default
  • compatibility-tools/tab
  • compatibility-tools/release-picker
  • logs/tab
  • settings/general
  • settings/cache-manager-modal
  • settings/language-selector
  • issue-reporting/template-selector
  • about/tab

High-Value Gaps To Add Next

  • Manage Configurations config editor modal
  • Manage Configurations ProtonDB submit modal
  • Manage Game missing-version decision modal
  • Manage Game installed-version picker modal
  • Compatibility Tools install-from-ZIP modal
  • Logs tab base view
  • Settings advanced cache/performance section

Publish Check

  • Retired screenshot keys do not remain in the wiki after publish
  • No duplicate PNGs exist for the same logical screen
  • Each screenshot lands on the correct visible page, not just the correct automation state
  • The gallery reads like a left-nav walkthrough before showing deeper modals

Sign-off

Area Pass Notes
Deployment
System detection
Sidebar panel
ProtonDB badge
Ranked report modal
Apply launch options
Logging
Game-running guard
Error handling
API field verification

Tester: Date: Steam client channel: Deck OS version: Decky Loader version: Plugin version:

Clone this wiki locally