Skip to content

Enhance test assertion failure messages with actual/expected values #803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 17, 2025

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Jun 3, 2025

The test functions in internal/provider/provider_test.go were using generic failure messages that made debugging difficult when assertions failed. This change replaces the generic messages with descriptive ones that include actual and expected values.

Changes Made

Before:

require.Equal(t, len(expectedDataSources), len(datasources), "There are an unexpected number of registered data sources")
require.Contains(t, expectedDataSources, d(), "An unexpected data source was registered")

After:

require.Equalf(t, len(expectedDataSources), len(datasources), "Expected %d data sources, got %d", len(expectedDataSources), len(datasources))
require.Containsf(t, expectedDataSources, d(), "Data source %+v was not expected", d())

Impact

When tests fail, developers will now see specific information like:

  • "Expected 25 data sources, got 24" instead of "There are an unexpected number of registered data sources"
  • "Data source {TypeName: example} was not expected" instead of "An unexpected data source was registered"

This provides immediate context about what went wrong, making debugging much faster and more efficient.

Fixes #802.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • checkpoint-api.hashicorp.com
    • Triggering command: /tmp/go-build1791408922/b001/provider.test -test.testlogfile=/tmp/go-build1791408922/b001/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true -test.run=TestUnit.* (dns block)

If you need me to access, download, or install something from one of these locations, you can either:


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: mawasile <50197777+mawasile@users.noreply.github.com>
@Copilot Copilot AI changed the title [WIP] [Copilot] Test Functions Lack Descriptive Failure Messages or Context Enhance test assertion failure messages with actual/expected values Jun 3, 2025
@Copilot Copilot AI requested a review from mawasile June 3, 2025 12:02
Copilot finished work on behalf of mawasile June 3, 2025 12:02
@mawasile mawasile marked this pull request as ready for review June 3, 2025 13:00
@Copilot Copilot AI review requested due to automatic review settings June 3, 2025 13:00
@mawasile mawasile requested a review from a team as a code owner June 3, 2025 13:00
@mawasile
Copy link
Contributor

mawasile commented Jun 3, 2025

@copilot add changelog

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves test failure messages by incorporating actual and expected values into assertion errors.

  • Test assertions in provider_test.go now use require.Equalf and require.Containsf with formatted messages
  • Error messages reveal specific discrepancies in data sources and resources counts
Comments suppressed due to low confidence (2)

internal/provider/provider_test.go:81

  • [nitpick] Consider rephrasing the error message to clarify that the data source was expected but not found; for example, 'Data source %+v not found in expected data sources' may improve readability.
require.Containsf(t, expectedDataSources, d(), "Data source %+v was not expected", d())

internal/provider/provider_test.go:113

  • [nitpick] Consider rephrasing the error message to better indicate that the resource was expected to be present but was missing; for example, 'Resource %+v not found in expected resources' would be clearer.
require.Containsf(t, expectedResources, r(), "Resource %+v was not expected", r())

Co-authored-by: mawasile <50197777+mawasile@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jun 3, 2025

@copilot add changelog

Added changelog entry in commit 6da0aa4. The change is categorized as a documentation improvement since it enhances test debugging visibility.

Copilot finished work on behalf of mawasile June 3, 2025 13:03
@mawasile mawasile added ai assisted Pull requests that have be fully or partially implemented using Copilot or AI ai found Issues and Bugs that were found using AI copilot fixed using GitHub copilot autonomous agent labels Jun 3, 2025
Co-authored-by: mattdot <266258+mattdot@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jun 4, 2025

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • http://168.63.129.16:80/machine/
    • Triggering command: /usr/bin/python3 -u bin/WALinuxAgent-2.13.1.1-py3.9.egg -collect-logs (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of mattdot June 4, 2025 19:13
@mawasile mawasile enabled auto-merge (squash) June 16, 2025 18:52
@mawasile mawasile merged commit fb68c2d into main Jun 17, 2025
19 of 20 checks passed
@mawasile mawasile deleted the copilot/fix-802 branch June 17, 2025 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ai assisted Pull requests that have be fully or partially implemented using Copilot or AI ai found Issues and Bugs that were found using AI copilot fixed using GitHub copilot autonomous agent skip-changelog
Projects
None yet
Development

Error loading sessions

Retrying...

Successfully merging this pull request may close these issues.

[Copilot] Test Functions Lack Descriptive Failure Messages or Context
4 participants