Skip to content

Implement CLI Support for Push AV Tests #59

Merged
rquidute merged 14 commits into
v2.15-cli-developfrom
feature/783_cli_support_push_av_tests
Jan 15, 2026
Merged

Implement CLI Support for Push AV Tests #59
rquidute merged 14 commits into
v2.15-cli-developfrom
feature/783_cli_support_push_av_tests

Conversation

@rquidute
Copy link
Copy Markdown
Contributor

@rquidute rquidute commented Jan 13, 2026

What changed

This PR implements CLI support for Push AV (Push Audio/Video) Stream Transport tests via the CLI.

How It Works

The CLI presents Push AV verification prompts using the same interface as the Test Harness browser UI:

  1. When a Push AV test runs, the CLI starts a local HTTP server
  2. The CLI displays a URL to the user (e.g., http://192.168.1.50:8999)
  3. User opens the URL in their browser to see the DASH player interface
  4. The browser displays the same verification UI as TH web interface
  5. User verifies the video stream and clicks PASS/FAIL
  6. The CLI forwards the response to complete the test

When executing in TH UI this is how the Push AV prompt is displayed:
Screenshot 2026-01-12 at 19 08 53

Now, the CLI will display a similar prompt like bellow:
Screenshot 2026-01-13 at 19 38 43

The cli execution log shows Push AV tests like bellow:
Screenshot 2026-01-13 at 19 40 32

Related Issue

project-chip/certification-tool#783

Testing

Unit test added and Passing
Screenshot 2026-01-13 at 19 13 38

Run a Push AV test successfully by running this command in CLI: th-cli run-tests -t TC_PAVSTI_1_2 --project-id 2

@rquidute rquidute self-assigned this Jan 13, 2026
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @rquidute, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates Push Audio/Video (AV) Stream Transport tests into the CLI workflow, providing a robust mechanism for users to execute and verify these tests without relying solely on the graphical user interface. It achieves this by introducing a local HTTP server that presents an interactive web-based prompt, allowing users to view and interact with streaming content, and then relaying their responses back to the CLI. This significantly enhances the automation and flexibility of Push AV testing.

Highlights

  • CLI Support for Push AV Tests: This pull request introduces comprehensive command-line interface (CLI) support for Push Audio/Video (AV) Stream Transport tests, enabling users to execute and verify these tests directly from the terminal.
  • Interactive Web-Based Prompt: A new interactive web-based prompt has been implemented for Push AV tests, mirroring the user experience of the existing UI. This prompt allows users to view available streams, play video content, and provide pass/fail responses.
  • HTTP Server Enhancements for Streaming: The CameraHTTPServer has been significantly enhanced to support Push AV streaming. This includes new endpoints for fetching stream lists (/api/streams), proxying stream data (/api/stream_proxy), and a simplified base64-encoded URL proxy (/proxy/). The server now dynamically rewrites DASH MPD manifests to ensure proper playback through the local proxy.
  • Dedicated HTML Template for Push AV Verification: A new HTML template, push_av_stream_verification.html, has been added. This template provides the user interface for interacting with Push AV streams, including stream selection, video playback via DASH.js, display of stream contents, and conformance status.
  • Expanded Prompt Manager Functionality: The prompt_manager module has been updated to handle PushAVStreamVerificationRequest and ImageVerificationPromptRequest types, routing them to appropriate handlers and integrating with the new HTTP server for interactive user input.
  • New Unit Tests: New unit tests have been added for camera_http_server and prompt_manager to ensure the reliability and correctness of the newly introduced Push AV CLI functionality and HTTP server enhancements.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request successfully implements CLI support for Push AV stream transport tests. It introduces a new web-based verification prompt for Push AV streams, complete with a DASH player and a proxy to handle stream fetching from the Push AV server. The changes are well-structured, with new handlers in prompt_manager.py and camera_http_server.py, a new HTML/JS interface, and corresponding unit tests.

My review focuses on improving security and test quality. I've pointed out the use of disabled SSL verification in httpx clients, which poses a security risk. I've also highlighted several placeholder unit tests that should be implemented to ensure the new validation logic is robust. A minor style issue regarding local imports has also been noted.

Overall, this is a solid implementation of a complex feature. Addressing the feedback will enhance the security and reliability of the new functionality.

Comment thread tests/test_run/test_prompt_manager.py Outdated
Comment thread tests/test_run/test_prompt_manager.py Outdated
Comment thread th_cli/test_run/camera/camera_http_server.py Outdated
Comment thread th_cli/test_run/camera/camera_http_server.py Outdated
Comment thread tests/test_run/__init__.py Outdated
Comment thread th_cli/test_run/prompt_manager.py Outdated
Comment thread th_cli/test_run/prompt_manager.py Outdated
Comment thread th_cli/test_run/prompt_manager.py Outdated
Comment thread tests/test_run/camera/__init__.py Outdated
@rquidute rquidute requested a review from antonio-amjr January 14, 2026 18:34
@rquidute rquidute merged commit ec1bd24 into v2.15-cli-develop Jan 15, 2026
2 checks passed
@rquidute rquidute deleted the feature/783_cli_support_push_av_tests branch January 15, 2026 09:21
oxesoft added a commit that referenced this pull request May 25, 2026
* [Fix] Handling Empty Usage Tag In CLI's PICS Parser (#43)

* PICS parser logic update: verifying if the 'usage' tag exists, ignoring if not.

* Gemini's suggestion: Improving conditional logic

* Removing unnecessary attribution line

* Fix _send_prompt_response calls and added timeout for large upload (#44)

* Fix _send_prompt_response calls and added timeout for large upload

* Update th_cli/test_run/prompt_manager.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review - Gemini

* Added constants values

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Show friendly message to unsupported TWO WAY TALK test execution using CLI (#46)

* Add Checking for  ffmpeg and improve abort-testing command

* Webrtc working

* Changes related to two way talk

* Updated ffmpeg parameters

* Deleted unwated config file

* changes after black command

* Update th_cli/commands/abort_testing.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/th_utils/ffmpeg_converter.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/test_run/websocket.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review gemini

* Code review gemini

* Removed extra space in warning message from shared_constant file

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Fixing the CLI Run Test Logic to handle project config and user config (#47)

* Fixing the CLI Run Test Logic to handle project config and user config

* Gemini Suggestions: Fixing Path creation and adding except print to the user.

* Removed Error column from test execution list (#49)

* [Feature] Adding the Node ID to CLI Output (#48)

* Adding the Node ID to CLI output

* Updating ChipServerInfo attribute type

Plus minor fixes

* Fix cli unit tests not working (#50)

* Fix CLI unit tests not working

* Removed Error column from test execution list (#49)

* Updated conftest and models

* Update th_cli/commands/test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [FIX] CLI PICS Parser And Prompt (#51)

* Adding Message Prompt Request type and fixing PICS XML parser

* Fixing some flake8 issues

* Fixing isort issue

* Gemini's Suggestion: Remove unnecessary timeout exception logic

* Fix test_run_execution command (#52)

* Fix test_run_execution command

* Update tests/test_test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/commands/test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Added support to project-id in test_run_execution command

* Black formating

* Code review

* Code review

* Code review

* code review

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [Feature] Printing Manual Pairing Code (#53)

* Printing the Manual Pairing Code to CLI's test execution output

* Gemini Suggestion: Improving logic and fixing f strings

* Fixing comment

Adding missing parameter for the method's comment

Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

---------

Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

* Adding the test parameters to the models to be used in the CLI's project config (#55)

Also, the logic to print the chipserver info was fixed to always show the Node ID, even without the manual pairing code

* Enhanced the th-cli available-tests command with new formatting options (#54)

* Improve CLI available-tests command

* Code formating

* Minor code changes

* Code review - gemini code assistant

* Increase websocket message size (#56)

* Increase websocket message size

* Create constant for WEBSOCKET_MAX_MESSAGE_SIZE

* Improve  FFmpeg Error Handling in Video Stream Tests (#57)

* Add message for ffmpeg not found

* Code review - gemini

* Code review

* Code review

* [Feature] Swap CLI Properties Config With JSON (#58)

* Swapping the Properties config file with the JSON format

* Fixing lint alerts

* New Line At File's End

Gemini Suggestion: adding a new line to the default_config.json file as per recommendation. This can prevent som e text processing tools errors.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Implement CLI Support for Push AV Tests  (#59)

* Initial code

* Stable version

* Added unit tests

* Minor changes

* Code formating and isort

* Updated unit test

* Code review

* Add comment regarding disabling SSL verification

* Removed unused import

* Removed unused imports

* Minor changes

* Fixed f-string

* Removed unecessary init files

* Code review - minor changes

* [Feature] Double-Dash Extra Arguments (#60)

* Implement the double dash extra arguments feature

Also including the Unit Tests

* Fixing ouput message

* Improving parser logic

* Fixing lint errors

* Improving parser logic for subsequent --

* Fixing the README Notes

* Fixing CLI run_tests command conditional for extra arguments

* Fixing ChipServer info display when using no project ID or config options (#61)

* Fix WebSocket close handshake error for long-running test executions (#63)

* Fix WebSocket close handshake error for long-running test executions in the CLI

* Update th_cli/test_run/websocket.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [FIX] Updating CLI API Client Generator (#62)

* Updating and Fixing CLI API Client Generator

Updating tool version, files, scripts to support ARM64 hosts and generate successfully

* Re-enable black formatter to fix several files structure

* Adding postprocessing script to fix return type of api_response.py file

Also, re-enabling the formatters in the end of the generation

* Returning the TimeoutException logic but now adding the import accordingly

* Updating shell scripts to handle host architecture type and new flag options

* Fixing licence header generation with mustache files

* Added NFC_WIFI and THREAD_MESHCOP

* [Fix] CLI Project Create and Default Config (#67)

* Fixing default config file and usage for project creation/update

Also, fixing script file extension for run_pytest

* Gemini Suggestion: Adding new line at end of file

* Update regex pattern for test ID validation (#66)

Edited the regex pattern, so that Tests such as TC-I-XXX work with the CLI, which threw an error before.

* [Feature] Replace OpenAPI Generator with Datamodel-Code-Generator (#65)

* New CLI API Client Generator

Exchanging OpenAPI-Generator with Datamodel-Code-Generator

* Removing Openapi-generator files and adding README

* Fixing parameter placeholder from project command

* Fixing the Project Update command to handle empty names

* Fixing Project Update Unit Tests

* Fixing script's README file

* Project update improvements and minor fixes

* Implementing some TODOs with missing logic for headers and multipart fields (file or data)

* Fixing the README title

* [Feature] Refactoring CLI Project Command (#68)

* Updating CLI project command to use Click's group feature with subcommads

* Simplifying logic using a SyncAPI generator and a contextmanager decorator

* Fix test case comparison logic (#69)

* Fix test case comparison logic

* Update th_cli/utils.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Rever code formating

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Add missing image_handler.py and image_verification.html for snapshot camera verification                                                (#71)

* Fixed Screenshot camera verification

* Fixed copyright year

* Code review Gemini-code-assist

* Code review Gemini-code-assist

* Fix regex pattern to support additional valid test ID formats   (#73)

* Fix regex to support single-segment and underscore-category test IDs (TC-I-X, TC-BR-X, TC-MCORE_FS-X.X)

* Update tests/test_validation.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Feature/909 Improve CLI unit tests code coverage for camera files (#72)

* Fixed Screenshot camera verification

* Fixed copyright year

* Code review Gemini-code-assist

* Code review Gemini-code-assist

* Improve CLI test coverage across camera files and minor static analysis violations

* Undo cov-fail-under change

* black formating

* [FIX] Run_Tests Command Configuration Logic (#74)

* Fixing run_tests command configuration logic

* Gemini review: improving logic

* CLI  to support two way talk tests (#75)

* Initial implementation to support two way talk

* Some code improvements

* Impreved the user experience by only resume with the test case execution only when the CLI detects that the browser was opened

* Improved code comments

* Update th_cli/test_run/camera/two_way_talk_handler.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/commands/run_tests.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review  - gemini code assist::Use dependency injection to pass the handler instance through the call stack

* Code review

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [Feature] Add PICS Parameter To CLI Projects (#78)

* Adding pics parameters to CLI's project commands

* Fixing project update unit test

* Updating run-test command to use the configured pics with none was provided

* Fetching default project when no project ID was provided

* Improving project manipulation in run-test command

* [Fix] Project Create Command With No PICS (#79)

* Fixing project create command when PICS is not provided

* Gemini Suggestion: Removing unecessary global variable. Using PICS object directly

* [FIX] Updating CLI To Handle Temporary PICS (#80)

* Updating CLI to handle PICS during execution to be temporary (not persisted to the project in DB)

* Fixing else conditional from run_tests.py file

* Fixing openapi.json Create Cli Test Run Execution description

* [FEATURE] Update CLI with grouped logs download feature (#82)

* Update CLI with grouped logs download feature

* allow endpoints to return binary data

* Fix(cli): unique per-run log filename with timestamp + mode=w (#984) (#83)

---------

Co-authored-by: antonio-amjr <116589331+antonio-amjr@users.noreply.github.com>
Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>
Co-authored-by: Antonio Melo Jr. <a_junior@apple.com>
Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>
Co-authored-by: ali gruenhaupt <yesilbas.ma@gmail.com>
Co-authored-by: Steven Green <sgreen@csa-iot.org>
Co-authored-by: abhisheksingh-esp <abhishek.singh@espressif.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants