Skip to content

Make CLI test failures verbose by not catching exceptions in the test runner#2220

Merged
Flix6x merged 3 commits into
mainfrom
copilot/fix-errors-in-tracking-capacity
Jun 3, 2026
Merged

Make CLI test failures verbose by not catching exceptions in the test runner#2220
Flix6x merged 3 commits into
mainfrom
copilot/fix-errors-in-tracking-capacity

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jun 2, 2026

When a CLI test fails, the default FlaskCliRunner swallows exceptions and surfaces only a non-zero exit code, forcing developers to hunt for the root cause manually.

Changes

  • flexmeasures/conftest.py: adds CustomFlaskCliRunner — a FlaskCliRunner subclass that defaults catch_exceptions=False — and sets it as test_cli_runner_class on the test app, so all existing app.test_cli_runner() calls get verbose exceptions for free with no per-test changes.
class CustomFlaskCliRunner(FlaskCliRunner):
    def invoke(self, *args, **kwargs):
        kwargs.setdefault("catch_exceptions", False)
        return super().invoke(*args, **kwargs)
  • flexmeasures/cli/tests/test_utils.py: adds a strict xfail test that invokes a command raising RuntimeError. It must fail (XFAIL); if exceptions start being silently caught again the test unexpectedly passes (XPASS), breaking the suite.

Copilot AI and others added 2 commits June 2, 2026 21:27
Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Copilot AI changed the title feat: add CustomFlaskCliRunner to propagate exceptions in CLI tests Make CLI test failures verbose by not catching exceptions in the test runner Jun 2, 2026
Copilot finished work on behalf of Flix6x June 2, 2026 21:28
Copilot AI requested a review from Flix6x June 2, 2026 21:28
@Flix6x Flix6x linked an issue Jun 2, 2026 that may be closed by this pull request
@Flix6x Flix6x marked this pull request as ready for review June 2, 2026 21:33
@Flix6x Flix6x added this to the 1.0.0 milestone Jun 2, 2026
…-tracking-capacity

# Conflicts:
#	flexmeasures/cli/tests/test_utils.py
@Flix6x Flix6x merged commit 62d6733 into main Jun 3, 2026
12 checks passed
@Flix6x Flix6x deleted the copilot/fix-errors-in-tracking-capacity branch June 3, 2026 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failing CLI tests are not verbose enough

2 participants