Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Test Results Summary for compatibility tests #53

Merged
merged 4 commits into from
Apr 20, 2023
Merged

Add Test Results Summary for compatibility tests #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
88193dd
Add GitHub Summary to present compatibility test results and filter t…
mpanchajanya Mar 30, 2023
0be8b96
Bump ginkgo and gomega versions
mpanchajanya Apr 17, 2023
2a0b9e8
Remove icons to table data
mpanchajanya Apr 19, 2023
c430950
Update the compatibility test command and description
mpanchajanya Apr 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 8 additions & 10 deletions .github/workflows/compatibility_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,17 @@ jobs:
make tools
echo "${PWD}/hack/tools/bin" >> $GITHUB_PATH
echo "${PWD}/bin" >> $GITHUB_PATH

- name: go cache
uses: actions/cache@v3
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
sudo apt-get update
sudo apt-get install -y jq curl

- name: Build Runtime Test Plugins
run: make build-compatibility-test-plugins

- name: Run Compatibility Tests
run: make run-compatibility-tests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a bit of duplication here and with the make run-compatibility-tests target. Was replacing the make target here just for testing, or is it the final iteration because it needs to do different things than the make target?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed the duplication using the same target

- name: Tests Results Summary
if: always()
run: |
TEST_RESULTS_MD=$(./hack/scripts/process-ginkgo-test-results.sh testresults/compatibility-tests.json)
echo "$TEST_RESULTS_MD" >> $GITHUB_STEP_SUMMARY
9 changes: 5 additions & 4 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,9 @@ GO_MODULES=$(shell find . -path "*/go.mod" | xargs -I _ dirname _)
## Compatibility Testing
## --------------------------------------

.PHONY: compatibility-tests
compatibility-tests: tools build-compatibility-test-plugins run-compatibility-tests ## Build and Run Compatibility tests

.PHONY: build-compatibility-test-plugins
build-compatibility-test-plugins: ## Builds all runtime compatibility test plugins
cd ./test/compatibility/testplugins && mkdir -p bin
Expand All @@ -136,9 +139,7 @@ build-compatibility-test-plugins: ## Builds all runtime compatibility test plugi
cd ./test/compatibility/testplugins/runtime-test-plugin-v0_28_0 && ${GO} mod tidy && GOOS=$(OS) GOARCH=$(ARCH) ${GO} build -o ../bin
cd ./test/compatibility/testplugins/runtime-test-plugin-latest && ${GO} mod tidy && GOOS=$(OS) GOARCH=$(ARCH) ${GO} build -o ../bin

# The below command runs the compatibility tests using ginkgo and filter the logs as per regex and exit code with '0' representing success and non-zero values indicating test failures
.PHONY: run-compatibility-tests
run-compatibility-tests: ## Run Compatibility tests
cd ./test/compatibility/framework/compatibilitytests && ${GINKGO} --keep-going --fail-fast --race -r ${COMPATIBILITY_TEST_VERBOSE} --randomize-all --trace --output-dir ./../../../../testresults --junit-report compatibility-tests.xml; \

.PHONY: compatibility-tests
compatibility-tests: tools build-compatibility-test-plugins run-compatibility-tests ## Build and Run Compatibility tests
${GINKGO} --keep-going --output-dir testresults --json-report=compatibility-tests.json --race ${GOTEST_VERBOSE} -r test/compatibility/framework/compatibilitytests --randomize-all --trace > /tmp/out && { cat /tmp/out | grep -Ev 'STEP:|seconds|.go:'; rm /tmp/out; } || { exit_code=$$?; cat /tmp/out | grep -Ev 'STEP:|seconds|.go:'; rm /tmp/out; exit $$exit_code; }
2 changes: 1 addition & 1 deletion command/deprecate_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (

"github.com/spf13/cobra"

. "github.com/onsi/ginkgo"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)

Expand Down
2 changes: 1 addition & 1 deletion component/reader_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"os"
"testing"

. "github.com/onsi/ginkgo"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)

Expand Down
21 changes: 11 additions & 10 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,15 @@ require (
github.com/logrusorgru/aurora v2.0.3+incompatible
github.com/mattn/go-isatty v0.0.11
github.com/olekukonko/tablewriter v0.0.5
github.com/onsi/ginkgo v1.16.5
github.com/onsi/gomega v1.27.2
github.com/onsi/ginkgo/v2 v2.9.2
github.com/onsi/gomega v1.27.6
github.com/pkg/errors v0.9.1
github.com/spf13/cobra v1.6.1
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.8.1
github.com/tj/assert v0.0.3
go.uber.org/multierr v1.8.0
golang.org/x/mod v0.8.0
golang.org/x/mod v0.9.0
golang.org/x/sync v0.1.0
gopkg.in/yaml.v3 v3.0.1
)
Expand All @@ -28,23 +28,24 @@ require (
github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/fatih/color v1.9.0 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/go-logr/logr v1.2.3 // indirect
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/mattn/go-colorable v0.1.4 // indirect
github.com/mattn/go-runewidth v0.0.9 // indirect
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b // indirect
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e // indirect
github.com/nxadm/tail v1.4.8 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
go.uber.org/atomic v1.7.0 // indirect
golang.org/x/net v0.7.0 // indirect
golang.org/x/sys v0.5.0 // indirect
golang.org/x/term v0.5.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/net v0.8.0 // indirect
golang.org/x/sys v0.6.0 // indirect
golang.org/x/term v0.6.0 // indirect
golang.org/x/text v0.8.0 // indirect
golang.org/x/tools v0.7.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
)
Loading