Skip to content

Enhanced the th-cli available-tests command with new formatting options#54

Merged
rquidute merged 4 commits into
v2.15-cli-developfrom
fix/822_th_cli_available-tests_too_large
Dec 31, 2025
Merged

Enhanced the th-cli available-tests command with new formatting options#54
rquidute merged 4 commits into
v2.15-cli-developfrom
fix/822_th_cli_available-tests_too_large

Conversation

@rquidute
Copy link
Copy Markdown
Contributor

What changed

Enhanced the th-cli available-tests command with new formatting options, improved pagination, and comprehensive cluster
extraction for better user experience when browsing Matter certification test cases.

New Command Options

  • --compact: Display test IDs only in a clean 4-per-line format with proper column spacing
  • --cluster : Filter tests by cluster name (case-insensitive) and show detailed information including ID, title,
    description, collection, and suite
  • --group-by-cluster: Group all test cases by cluster with organized headers and 4-per-line formatting

Improved Pagination

  • Replaced verbose output with Click's echo_via_pager for git log-style navigation (Enter=1 line, Space=page, q=quit)
  • Applied pagination to all output formats (YAML, JSON, compact, grouped)
  • Maintains original YAML/JSON formatting for default behavior

Related Issue

project-chip/certification-tool#822

Testing

Test Coverage

  • Added unit test and they are all passing

Testing the following commands successfully:

  • th-cli available-tests --cluster WEBRTCR
  • th-cli available-tests --group-by-cluster
  • th-cli available-tests --json > cluster_json.json
  • th-cli available-tests | grep WEBRTCP
  • th-cli available-tests --compact | grep TC_WEBRTC_1_6

@rquidute rquidute self-assigned this Dec 30, 2025
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @rquidute, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the th-cli available-tests command by introducing flexible new output formatting options and improving the user experience with better pagination. Users can now view test cases in a compact ID-only format, filter by specific clusters with detailed information, or group all tests by their respective clusters, all while benefiting from an interactive pager for easier navigation through large lists.

Highlights

  • New Formatting Options: Introduced three new command-line options for the available-tests command: --compact for displaying only test IDs in a clean 4-per-line format, --cluster <NAME> for filtering tests by a specific cluster name (case-insensitive) and showing detailed information, and --group-by-cluster to organize all test cases by their respective clusters with 4-per-line formatting.
  • Improved Pagination: Integrated click.echo_via_pager across all output formats (YAML, JSON, compact, grouped, and cluster-filtered). This replaces verbose output with a git log-style navigation experience, allowing users to easily browse large lists of test cases.
  • Enhanced Cluster Extraction: Implemented robust logic to accurately extract cluster names from various test ID patterns, ensuring precise filtering and grouping capabilities for Matter certification test cases.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the th-cli available-tests command with new formatting options (--compact, --cluster, --group-by-cluster), pagination, and cluster extraction logic. The changes are well-tested with new unit tests. My review focuses on improving the implementation of the new helper functions for better maintainability and correctness. I've identified some unreachable code, code duplication, redundant regex patterns, and an opportunity to use a standard library module for text wrapping.

Comment thread th_cli/commands/available_tests.py Outdated
Comment thread th_cli/commands/available_tests.py Outdated
Comment thread th_cli/commands/available_tests.py Outdated
Comment thread th_cli/commands/available_tests.py Outdated
@rquidute rquidute merged commit 5415ccb into v2.15-cli-develop Dec 31, 2025
2 checks passed
@rquidute rquidute deleted the fix/822_th_cli_available-tests_too_large branch December 31, 2025 15:23
oxesoft added a commit that referenced this pull request May 25, 2026
* [Fix] Handling Empty Usage Tag In CLI's PICS Parser (#43)

* PICS parser logic update: verifying if the 'usage' tag exists, ignoring if not.

* Gemini's suggestion: Improving conditional logic

* Removing unnecessary attribution line

* Fix _send_prompt_response calls and added timeout for large upload (#44)

* Fix _send_prompt_response calls and added timeout for large upload

* Update th_cli/test_run/prompt_manager.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review - Gemini

* Added constants values

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Show friendly message to unsupported TWO WAY TALK test execution using CLI (#46)

* Add Checking for  ffmpeg and improve abort-testing command

* Webrtc working

* Changes related to two way talk

* Updated ffmpeg parameters

* Deleted unwated config file

* changes after black command

* Update th_cli/commands/abort_testing.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/th_utils/ffmpeg_converter.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/test_run/websocket.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review gemini

* Code review gemini

* Removed extra space in warning message from shared_constant file

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Fixing the CLI Run Test Logic to handle project config and user config (#47)

* Fixing the CLI Run Test Logic to handle project config and user config

* Gemini Suggestions: Fixing Path creation and adding except print to the user.

* Removed Error column from test execution list (#49)

* [Feature] Adding the Node ID to CLI Output (#48)

* Adding the Node ID to CLI output

* Updating ChipServerInfo attribute type

Plus minor fixes

* Fix cli unit tests not working (#50)

* Fix CLI unit tests not working

* Removed Error column from test execution list (#49)

* Updated conftest and models

* Update th_cli/commands/test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [FIX] CLI PICS Parser And Prompt (#51)

* Adding Message Prompt Request type and fixing PICS XML parser

* Fixing some flake8 issues

* Fixing isort issue

* Gemini's Suggestion: Remove unnecessary timeout exception logic

* Fix test_run_execution command (#52)

* Fix test_run_execution command

* Update tests/test_test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update tests/test_test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/commands/test_run_execution.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Added support to project-id in test_run_execution command

* Black formating

* Code review

* Code review

* Code review

* code review

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [Feature] Printing Manual Pairing Code (#53)

* Printing the Manual Pairing Code to CLI's test execution output

* Gemini Suggestion: Improving logic and fixing f strings

* Fixing comment

Adding missing parameter for the method's comment

Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

---------

Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>

* Adding the test parameters to the models to be used in the CLI's project config (#55)

Also, the logic to print the chipserver info was fixed to always show the Node ID, even without the manual pairing code

* Enhanced the th-cli available-tests command with new formatting options (#54)

* Improve CLI available-tests command

* Code formating

* Minor code changes

* Code review - gemini code assistant

* Increase websocket message size (#56)

* Increase websocket message size

* Create constant for WEBSOCKET_MAX_MESSAGE_SIZE

* Improve  FFmpeg Error Handling in Video Stream Tests (#57)

* Add message for ffmpeg not found

* Code review - gemini

* Code review

* Code review

* [Feature] Swap CLI Properties Config With JSON (#58)

* Swapping the Properties config file with the JSON format

* Fixing lint alerts

* New Line At File's End

Gemini Suggestion: adding a new line to the default_config.json file as per recommendation. This can prevent som e text processing tools errors.

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Implement CLI Support for Push AV Tests  (#59)

* Initial code

* Stable version

* Added unit tests

* Minor changes

* Code formating and isort

* Updated unit test

* Code review

* Add comment regarding disabling SSL verification

* Removed unused import

* Removed unused imports

* Minor changes

* Fixed f-string

* Removed unecessary init files

* Code review - minor changes

* [Feature] Double-Dash Extra Arguments (#60)

* Implement the double dash extra arguments feature

Also including the Unit Tests

* Fixing ouput message

* Improving parser logic

* Fixing lint errors

* Improving parser logic for subsequent --

* Fixing the README Notes

* Fixing CLI run_tests command conditional for extra arguments

* Fixing ChipServer info display when using no project ID or config options (#61)

* Fix WebSocket close handshake error for long-running test executions (#63)

* Fix WebSocket close handshake error for long-running test executions in the CLI

* Update th_cli/test_run/websocket.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [FIX] Updating CLI API Client Generator (#62)

* Updating and Fixing CLI API Client Generator

Updating tool version, files, scripts to support ARM64 hosts and generate successfully

* Re-enable black formatter to fix several files structure

* Adding postprocessing script to fix return type of api_response.py file

Also, re-enabling the formatters in the end of the generation

* Returning the TimeoutException logic but now adding the import accordingly

* Updating shell scripts to handle host architecture type and new flag options

* Fixing licence header generation with mustache files

* Added NFC_WIFI and THREAD_MESHCOP

* [Fix] CLI Project Create and Default Config (#67)

* Fixing default config file and usage for project creation/update

Also, fixing script file extension for run_pytest

* Gemini Suggestion: Adding new line at end of file

* Update regex pattern for test ID validation (#66)

Edited the regex pattern, so that Tests such as TC-I-XXX work with the CLI, which threw an error before.

* [Feature] Replace OpenAPI Generator with Datamodel-Code-Generator (#65)

* New CLI API Client Generator

Exchanging OpenAPI-Generator with Datamodel-Code-Generator

* Removing Openapi-generator files and adding README

* Fixing parameter placeholder from project command

* Fixing the Project Update command to handle empty names

* Fixing Project Update Unit Tests

* Fixing script's README file

* Project update improvements and minor fixes

* Implementing some TODOs with missing logic for headers and multipart fields (file or data)

* Fixing the README title

* [Feature] Refactoring CLI Project Command (#68)

* Updating CLI project command to use Click's group feature with subcommads

* Simplifying logic using a SyncAPI generator and a contextmanager decorator

* Fix test case comparison logic (#69)

* Fix test case comparison logic

* Update th_cli/utils.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Rever code formating

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Add missing image_handler.py and image_verification.html for snapshot camera verification                                                (#71)

* Fixed Screenshot camera verification

* Fixed copyright year

* Code review Gemini-code-assist

* Code review Gemini-code-assist

* Fix regex pattern to support additional valid test ID formats   (#73)

* Fix regex to support single-segment and underscore-category test IDs (TC-I-X, TC-BR-X, TC-MCORE_FS-X.X)

* Update tests/test_validation.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Feature/909 Improve CLI unit tests code coverage for camera files (#72)

* Fixed Screenshot camera verification

* Fixed copyright year

* Code review Gemini-code-assist

* Code review Gemini-code-assist

* Improve CLI test coverage across camera files and minor static analysis violations

* Undo cov-fail-under change

* black formating

* [FIX] Run_Tests Command Configuration Logic (#74)

* Fixing run_tests command configuration logic

* Gemini review: improving logic

* CLI  to support two way talk tests (#75)

* Initial implementation to support two way talk

* Some code improvements

* Impreved the user experience by only resume with the test case execution only when the CLI detects that the browser was opened

* Improved code comments

* Update th_cli/test_run/camera/two_way_talk_handler.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Update th_cli/commands/run_tests.py

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* Code review  - gemini code assist::Use dependency injection to pass the handler instance through the call stack

* Code review

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* [Feature] Add PICS Parameter To CLI Projects (#78)

* Adding pics parameters to CLI's project commands

* Fixing project update unit test

* Updating run-test command to use the configured pics with none was provided

* Fetching default project when no project ID was provided

* Improving project manipulation in run-test command

* [Fix] Project Create Command With No PICS (#79)

* Fixing project create command when PICS is not provided

* Gemini Suggestion: Removing unecessary global variable. Using PICS object directly

* [FIX] Updating CLI To Handle Temporary PICS (#80)

* Updating CLI to handle PICS during execution to be temporary (not persisted to the project in DB)

* Fixing else conditional from run_tests.py file

* Fixing openapi.json Create Cli Test Run Execution description

* [FEATURE] Update CLI with grouped logs download feature (#82)

* Update CLI with grouped logs download feature

* allow endpoints to return binary data

* Fix(cli): unique per-run log filename with timestamp + mode=w (#984) (#83)

---------

Co-authored-by: antonio-amjr <116589331+antonio-amjr@users.noreply.github.com>
Co-authored-by: Romulo Quidute Filho <116586593+rquidute@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Carolina Lopes <116589288+ccruzagralopes@users.noreply.github.com>
Co-authored-by: Antonio Melo Jr. <a_junior@apple.com>
Co-authored-by: Romulo Quidute Filho <rquidute@apple.com>
Co-authored-by: ali gruenhaupt <yesilbas.ma@gmail.com>
Co-authored-by: Steven Green <sgreen@csa-iot.org>
Co-authored-by: abhisheksingh-esp <abhishek.singh@espressif.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants