Fix WebSocket close handshake error for long-running test executions

[rquidute]

What changed

Fix WebSocket close handshake error for long-running test executions in the CLI. This change prevents the error message "sent 1000 (OK); no close frame received" from appearing after successful test completion.

Changes

WebSocket Connection Management:

• Added close_timeout=10 parameter to websocket_connect() to allow 10 seconds for the close handshake to complete
• Wrapped the entire async with websocket_connect() context manager in a try-except block to catch ConnectionClosedError and ConnectionClosed exceptions
• Added exception handling around the explicit socket.close() call in __handle_test_update() method

Exception Handling:

• Gracefully suppresses WebSocket close handshake errors that occur when the backend doesn't respond with a close frame
• Added inline comments explaining that these errors are expected with long-running test executions

Motivation

Problem:
When running long test executions (e.g., TC-ACE-2.1) via the CLI, users would see the error message "Error: Unexpected error during test execution: sent 1000 (OK); no close frame received" even though the test completed successfully and showed as PASSED. This error did not occur when running tests through the TH UI.

Root Cause:

• During long-running test executions, the backend server sometimes fails to complete the WebSocket close handshake properly
• When the CLI sends a close frame (status code 1000 - normal closure), the backend may not respond with its own close frame
• This causes the WebSocket library to raise a ConnectionClosedError exception, which was propagating up to the user

Solution:

• Added a close_timeout to give the backend more time to respond during the close handshake
• Added exception handling to suppress the error when the backend doesn't complete the handshake properly, since this is a cosmetic issue that doesn't affect test results

Impact

User Experience:

• ✅ Tests will complete cleanly without confusing error messages
• ✅ Test results remain accurate and show correct PASS/FAIL status
• ✅ No breaking changes to existing functionality

Technical:

• The 10-second close timeout is reasonable for long-running tests and won't impact performance
• Exception handling is specific to connection close errors and won't mask other legitimate errors
• Changes are isolated to the WebSocket connection handling code

Related Issue

project-chip/certification-tool#866

Testing

• Verified with TC-ACE-2.1, 2.2 & 2.4 test execution
• Error message no longer appears while test results remain accurate

[gemini-code-assist]

Summary of Changes

Hello @rquidute, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the stability and user experience of the CLI's test execution by addressing a specific WebSocket close handshake error. It introduces more resilient connection management and error handling for WebSocket connections, particularly during prolonged test runs, to prevent misleading error messages from appearing when tests have actually completed successfully.

Highlights

• WebSocket Connection Management: Added a close_timeout=10 parameter to the websocket_connect() function to allow 10 seconds for the close handshake to complete, preventing premature connection closure issues.
• Robust Error Handling: Implemented a try-except block around the websocket_connect() context manager to gracefully catch ConnectionClosedError and ConnectionClosed exceptions, which occur when the backend fails to send a close frame during long-running tests.
• Explicit Socket Closure Handling: Wrapped the explicit socket.close() call within the __handle_test_update() method in a try-except block to suppress ConnectionClosedError when the backend closes the connection without completing the handshake, as this is an expected scenario for successful test completions.
• Improved User Experience: The changes eliminate the confusing 'sent 1000 (OK); no close frame received' error message that previously appeared after successful long-running test executions in the CLI, ensuring a cleaner output for users.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• th_cli/test_run/websocket.py
  • Added close_timeout=10 to the websocket_connect call to provide a grace period for the WebSocket close handshake.
  • Wrapped the async with websocket_connect(...) block in a try-except statement to catch websockets.exceptions.ConnectionClosedError and websockets.exceptions.ConnectionClosed, suppressing these errors when the backend doesn't complete the close handshake.
  • Added a try-except block around await socket.close() in __handle_test_update to handle websockets.exceptions.ConnectionClosedError gracefully, acknowledging that the backend might close the connection without a full handshake after a test completes.

Activity

• The pull request addresses the issue described in [CLI] TC-ACE-2.1, 2.2 & 2.4 – Tests pass but unexpected “no close frame received” error after execution certification-tool#866.
• The author verified the fix by running TC-ACE-2.1, 2.2, and 2.4 test executions.
• Verification confirmed that the error message no longer appears, and test results remain accurate.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses a WebSocket close handshake error that occurs during long-running test executions. The changes involve adding a close_timeout to the WebSocket connection and introducing try-except blocks to gracefully handle ConnectionClosedError exceptions when closing the socket. This prevents a cosmetic error message from being displayed to the user upon successful test completion. My review focuses on the correctness of the exception handling. The changes are well-targeted to solve the described problem, and I have one suggestion to simplify the exception handling logic for better readability and correctness.