Cross-version API Compatibility - Follow up changes on test results and additional tests #26
Conversation
mpanchajanya
force-pushed
the
compatibility_tests_results
branch
12 times, most recently
from
0f2b32f
to
51ec95e
Compare
…ts involving multiple context objects
Makefile
Outdated
| cd ./test/compatibility/framework/compatibilitytests && ${GINKGO} --keep-going --fail-fast --race -r --randomize-all --trace --output-dir ./../../../../testresults --junit-report compatibility-tests.xml; \ | ||
|
|
||
| .PHONY: debug-compatibility-tests | ||
| debug-compatibility-tests: ## Run Compatibility tests |
There was a problem hiding this comment.
My preference, to reduce duplication, and so the developer doesn't have to deal with multiple targets, is to use a variable to control -v or not, not unlike what is used in https://github.com/vmware-tanzu/tanzu-cli/blob/main/Makefile#L198
There was a problem hiding this comment.
Updated the scripts
mpanchajanya
changed the title
mpanchajanya
force-pushed
the
compatibility_tests_results
branch
from
c62dbdd
to
3798ffe
Compare
Makefile
Outdated
| context-api-compatibility-tests: ## Run Compatibility tests | ||
| ${GO} test `go list ./test/compatibility/framework/compatibilitytests/context...` -v -timeout 60m -race;\ | ||
|
|
||
| cd ./test/compatibility/framework/compatibilitytests && ${GINKGO} --keep-going --fail-fast --race -r --randomize-all --trace --output-dir ./../../../../testresults --junit-report compatibility-tests.xml; \ |
There was a problem hiding this comment.
what happened to the equivalent of debug-compatibility-tests?
There was a problem hiding this comment.
added debug-compatibility-tests with ${GOTEST_VERBOSE}
There was a problem hiding this comment.
What I meant, in an earlier comment, was to update run-compatibility-tests to take a flag that is configurable.
If that flag is not ${GOTEST_VERBOSE} because the default value does not match the default behavior, then create one for the compat tests.
mpanchajanya
force-pushed
the
compatibility_tests_results
branch
from
3798ffe
to
b0caf60
Compare
| | Client Options APIs | TBA | TBA | TBA | TBA | | ||
| | Repositories APIs | TBA | TBA | TBA | TBA | | ||
|
|
||
| - For the latest test results check the most recent job that is run for the workflow [Tanzu Plugin Runtime Cross-version API Compatibility Tests](https://github.com/vmware-tanzu/tanzu-plugin-runtime/actions/workflows/compatibility_tests.yaml?query=branch%3Amain) |
There was a problem hiding this comment.
let see as a followup if we can make the results more accessible. Even if I bookmark the link it takes 4 clicks just to get to what the results.
There was a problem hiding this comment.
Added a JIRA to track this changes. https://jira.eng.vmware.com/browse/TANZUCLI-409
|
|
||
| - For the latest test results check the most recent job that is run for the workflow [Tanzu Plugin Runtime Cross-version API Compatibility Tests](https://github.com/vmware-tanzu/tanzu-plugin-runtime/actions/workflows/compatibility_tests.yaml?query=branch%3Amain) | ||
|
|
||
| *TBA - to be added in future sprints |
mpanchajanya
force-pushed
the
compatibility_tests_results
branch
from
1b4bf22
to
8711568
Compare
… flag in run-compatibility-tests
mpanchajanya
force-pushed
the
compatibility_tests_results
branch
from
8711568
to
efe6353
Compare
What this PR does / why we need it
Follow up PR addresses
Which issue(s) this PR fixes
Fixes #
Describe testing done for PR
make compatibility-testsOutput:
More details on test output can be found from the Github workflow that is run on the PR https://github.com/vmware-tanzu/tanzu-plugin-runtime/actions/runs/4495795430/jobs/7909811037?pr=26
For more details on Cross-version API compatibility testing refer to https://github.com/vmware-tanzu/tanzu-plugin-runtime/blob/main/test/compatibility/docs/cross-version-api-compatibility.md
Release note
Additional information
Special notes for your reviewer