Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand UI test #1027

Merged
merged 50 commits into from
Nov 20, 2021
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
1858d9a
ui test refactoring
Nov 4, 2021
60d9df4
lint check
Nov 4, 2021
37d473f
fix lint problems
Nov 4, 2021
ae788db
store acceptance test artifacts
Nov 4, 2021
c79bea0
set env for base dir
Nov 4, 2021
204492a
check dir
Nov 4, 2021
8bf0bfc
remove needless input field
Nov 4, 2021
a62a70f
save screenshots
Nov 4, 2021
1279807
fix for nightly
Nov 7, 2021
efcfba4
stop cuddling if statements
Nov 7, 2021
598c457
fix element not found issue
Nov 8, 2021
2057ea8
fix lint issue
Nov 8, 2021
6993c7d
skip tag for quick run
Nov 9, 2021
356d765
Merge branch 'main' into expand-ui-tests
rokshana-b Nov 9, 2021
01a93d0
fix lint error
Nov 9, 2021
fb276b5
fix string error
Nov 9, 2021
1cfd571
refactor test case
Nov 10, 2021
b3a4b48
makes changes for new directory structure
Nov 10, 2021
825c1a2
Merge branch 'main' into expand-ui-tests
Nov 10, 2021
8c11988
fix lint issue
Nov 10, 2021
9a3c622
remove skip tags
Nov 10, 2021
e88826f
fix instatlation step
Nov 10, 2021
d479135
Merge branch 'main' into expand-ui-tests
rokshana-b Nov 16, 2021
3f895f2
Update test/acceptance/test/ui_tests.go
rokshana-b Nov 16, 2021
8b23bac
separate ui tests and store screenshot
Nov 16, 2021
3115d35
Revert "separate ui tests and store screenshot"
Nov 16, 2021
efc21ec
Merge branch 'main' into expand-ui-tests
Nov 16, 2021
f23bc6f
Merge branch 'expand-ui-tests' of github.com:weaveworks/weave-gitops …
Nov 16, 2021
587f014
resolve conflict
Nov 16, 2021
4c045a0
separate selenium settings
Nov 16, 2021
67d1fdb
use selenium action
Nov 16, 2021
7eef300
skip selenium setting for cli tests
Nov 16, 2021
d42cff9
remove file
Nov 16, 2021
fee9e0a
refactor
Nov 16, 2021
62b765f
try out syntax
Nov 16, 2021
224d0e1
Merge branch 'main' into expand-ui-tests
Nov 18, 2021
93ad4e7
Merge branch 'main' into expand-ui-tests
Nov 18, 2021
c4b7205
refine xpath
Nov 18, 2021
6d9b004
expand tests
Nov 19, 2021
e62cbc3
fix lint
Nov 19, 2021
a257cc3
lint issues
Nov 19, 2021
3008c2f
use better xpath
Nov 19, 2021
bc71139
try to find element
Nov 19, 2021
17333d5
fix typo
Nov 19, 2021
f53e7cb
fix test
Nov 19, 2021
8777ebb
remove focus tag
Nov 19, 2021
f2986e4
remove verification for git auth
Nov 19, 2021
e53dcf5
address requested changes
Nov 19, 2021
c726c6c
try to store test results even after failure
Nov 19, 2021
68c4dcb
fix yaml lint
Nov 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/actions/run-acceptance-test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ inputs:
gitlab-token:
description: "Token for accessing GitLab"
required: true
artifacts-base-dir:
description: "Directory for test artifacts"
required: true

runs:
using: "composite"
Expand All @@ -31,6 +34,7 @@ runs:
- name: Checkout code
uses: actions/checkout@v2
- name: Setup selenium standalone server
if: ${{ github.job == 'ui-tests' }}
shell: bash
run: |
sudo apt-get update
Expand Down Expand Up @@ -91,6 +95,7 @@ runs:
GITLAB_SUBGROUP: weave-gitops-sub
GITLAB_TOKEN: ${{ inputs.gitlab-token }}
GITLAB_KEY: ${{ inputs.gitlab-key }}
ARTIFACTS_BASE_DIR: ${{ inputs.artifacts-base-dir }}
run: |
export PATH=${PATH}:`go env GOPATH`/bin
export WEGO_BIN_PATH=$(pwd)/bin/gitops
Expand All @@ -100,7 +105,7 @@ runs:
ginkgo ${{ inputs.focus-or-skip }} --randomizeSuites --reportFile=${{ env.ARTIFACTS_BASE_DIR }}/${{ inputs.result-prefix }}acceptance-test-results.xml -v ./test/acceptance/test/...
- name: Store acceptance test results
env:
ARTIFACTS_BASE_DIR: "/tmp/wego-test"
ARTIFACTS_BASE_DIR: "/tmp/gitops-test"
rokshana-b marked this conversation as resolved.
Show resolved Hide resolved
uses: actions/upload-artifact@v2
with:
name: ${{ inputs.result-prefix }}acceptance-test-artifacts
Expand Down
24 changes: 23 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ jobs:
github-token: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_TOKEN }}
gitlab-key: ${{ secrets.GITLAB_KEY }}
gitlab-token: ${{ secrets.GITLAB_TOKEN }}
artifacts-base-dir: "/tmp/gitops-test"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This value is used in a whole bunch of jobs. I think it should be moved to the environment for the workflow:

name: run tests
env:
  ARTIFACTS_BASE_DIR: "/tmp/gitops-test"
jobs:
...
       artifacts-base-dir: ${{ env.ARTIFACTS_BASE_DIR }}
...


smoke-tests-long:
runs-on: ubuntu-latest
Expand All @@ -142,6 +143,7 @@ jobs:
github-token: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_TOKEN }}
gitlab-key: ${{ secrets.GITLAB_KEY }}
gitlab-token: ${{ secrets.GITLAB_TOKEN }}
artifacts-base-dir: "/tmp/gitops-test"

acceptance-tests-0:
runs-on: ubuntu-latest
Expand All @@ -151,11 +153,12 @@ jobs:
- uses: actions/checkout@v2
- uses: ./.github/actions/run-acceptance-test
with:
focus-or-skip: "--skip='Test1|Test2|Test3|SmokeTest|SmokeTestLong'"
focus-or-skip: "--skip='Test1|Test2|Test3|SmokeTest|SmokeTestLong|UITest'"
github-key: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_SSH_KEY }}
github-token: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_TOKEN }}
gitlab-key: ${{ secrets.GITLAB_KEY }}
gitlab-token: ${{ secrets.GITLAB_TOKEN }}
artifacts-base-dir: "/tmp/gitops-test"

acceptance-tests-1:
runs-on: ubuntu-latest
Expand All @@ -171,6 +174,7 @@ jobs:
github-token: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_TOKEN }}
gitlab-key: ${{ secrets.GITLAB_KEY }}
gitlab-token: ${{ secrets.GITLAB_TOKEN }}
artifacts-base-dir: "/tmp/gitops-test"

acceptance-tests-2:
runs-on: ubuntu-latest
Expand All @@ -186,6 +190,7 @@ jobs:
github-token: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_TOKEN }}
gitlab-key: ${{ secrets.GITLAB_KEY }}
gitlab-token: ${{ secrets.GITLAB_TOKEN }}
artifacts-base-dir: "/tmp/gitops-test"

acceptance-tests-3:
runs-on: ubuntu-latest
Expand All @@ -201,6 +206,23 @@ jobs:
github-token: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_TOKEN }}
gitlab-key: ${{ secrets.GITLAB_KEY }}
gitlab-token: ${{ secrets.GITLAB_TOKEN }}
artifacts-base-dir: "/tmp/gitops-test"

ui-tests:
runs-on: ubuntu-latest
needs: [lint, build]
timeout-minutes: 90
steps:
- uses: actions/checkout@v2
- uses: ./.github/actions/run-acceptance-test
with:
result-prefix: 'ui-test-'
focus-or-skip: "--focus='UITest'"
github-key: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_SSH_KEY }}
github-token: ${{ secrets.WEAVE_GITOPS_TEST_WEAVEWORKS_WEAVE_GITOPS_BOT_TOKEN }}
gitlab-key: ${{ secrets.GITLAB_KEY }}
gitlab-token: ${{ secrets.GITLAB_TOKEN }}
artifacts-base-dir: "/tmp/gitops-test"

library-integration-test:
runs-on: ubuntu-latest
Expand Down
4 changes: 2 additions & 2 deletions test/acceptance/test/add_tests.go
Original file line number Diff line number Diff line change
Expand Up @@ -325,8 +325,8 @@ var _ = Describe("Weave GitOps Add App Tests", func() {
tip := generateTestInputs()
appName := tip.appRepoName
appConfigRepoName := "wego-config-repo-" + RandString(8)
appRepoRemoteURL := "ssh://git@github.com/" + GITHUB_ORG + "/" + tip.appRepoName + ".git"
configRepoRemoteURL = "ssh://git@github.com/" + GITHUB_ORG + "/" + appConfigRepoName + ".git"
appRepoRemoteURL := "https://github.com/" + GITHUB_ORG + "/" + tip.appRepoName + ".git"
rokshana-b marked this conversation as resolved.
Show resolved Hide resolved
configRepoRemoteURL = "https://github.com/" + GITHUB_ORG + "/" + appConfigRepoName + ".git"

addCommand := "add app --url=" + appRepoRemoteURL + " --app-config-url=" + configRepoRemoteURL + " --auto-merge=true"

Expand Down
34 changes: 34 additions & 0 deletions test/acceptance/test/pages/add_application_page.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package pages

import (
"github.com/sclevine/agouti"
)

type AddAppPageElements struct {
AddAppHeader *agouti.Selection
AppName *agouti.Selection
AppNamespace *agouti.Selection
AppRepoURL *agouti.Selection
ConfigRepoURL *agouti.Selection
PathToManifests *agouti.Selection
Branch *agouti.Selection
AutoMergeCheck *agouti.Selection
SubmitButton *agouti.Selection
AuthenticationButton *agouti.Selection
}

func GetAddAppPageElements(webDriver *agouti.Page) *AddAppPageElements {
addApplicationPage := AddAppPageElements{
AddAppHeader: webDriver.FindByXPath(`//*[@id="app"]//nav//h2[text()="Add Application"]`),
AppName: webDriver.FindByID("name"),
AppNamespace: webDriver.FindByID("namespace"),
AppRepoURL: webDriver.FindByID("url"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be very brittle.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you plan to change the ID placement a lot? Or will the overall UI outlook change too much?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the xpaths to be as unique as possible

ConfigRepoURL: webDriver.FindByID("configUrl"),
PathToManifests: webDriver.FindByID("path"),
Branch: webDriver.FindByID("branch"),
AutoMergeCheck: webDriver.FindByXPath(`//input[@type="checkbox" and @class="jss12 MuiSwitch-input"]`),
jpellizzari marked this conversation as resolved.
Show resolved Hide resolved
SubmitButton: webDriver.FindByXPath(`//button[@type="submit" and //@span=(text()="Submit")]`),
AuthenticationButton: webDriver.FindByXPath(`//button[@type="button" and @span=(text()="Authenticate with Github")]`)}

return &addApplicationPage
}
67 changes: 67 additions & 0 deletions test/acceptance/test/pages/application_page.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
package pages

import "github.com/sclevine/agouti"

type AppDetailsPageElements struct {
ApplicationsHeader *agouti.Selection
NameSubheader *agouti.Selection
DeploymentTypeSubheader *agouti.Selection
URLSubheader *agouti.Selection
PathSubheader *agouti.Selection
AppName *agouti.Selection
}

type NamedElements struct {
AppNameHeader *agouti.Selection
AppName *agouti.Selection
AppType *agouti.Selection
AppURL *agouti.Selection
AppPathToManifests *agouti.Selection
HelmSuccessMessage *agouti.Selection
KustomizeSuccessMessage *agouti.Selection
}

func GetAppDetailsPageElements(webDriver *agouti.Page) *AppDetailsPageElements {
appDetailsPage := AppDetailsPageElements{
ApplicationsHeader: webDriver.FindByXPath(`//*[@id="app"]//nav/ol/li/a/span/h2[text()="Applications"]`),
NameSubheader: webDriver.FindByXPath(`//table/tbody/tr/td[1]/div[text()="Name"]`),
DeploymentTypeSubheader: webDriver.FindByXPath(`//table/tbody/tr/td[2]/div[text()="Deployment Type"]`),
URLSubheader: webDriver.FindByXPath(`//table/tbody/tr/td[3]/div[text()="URL"]`),
PathSubheader: webDriver.FindByXPath(`//table/tbody/tr/td[4]/div[text()="Path"]`)}

return &appDetailsPage
}

func GetAppNameElements(webDriver *agouti.Page, appName string) *NamedElements {
appNameElements := NamedElements{
AppNameHeader: webDriver.FindByXPath(`//*[@id="app"]//div/nav/ol/li/h2[text()="` + appName + `"]`),
AppName: webDriver.FindByXPath(`//table/tbody/tr/td[1]/div[text()="` + appName + `"]`)}

return &appNameElements
}

func GetAppTypeElement(webDriver *agouti.Page, appType string) *NamedElements {
appTypeElement := NamedElements{
AppType: webDriver.FindByXPath(`//table/tbody/tr/td[2]/div[text()="` + appType + `"]`)}
return &appTypeElement
}

func GetURLElement(webDriver *agouti.Page, appURL string) *NamedElements {
appURLElement := NamedElements{
AppURL: webDriver.FindByXPath(`//table/tbody/tr/td[3]/div[text()="` + appURL + `"]`)}
return &appURLElement
}

func GetPathElement(webDriver *agouti.Page, pathToManifests string) *NamedElements {
appPathElement := NamedElements{
AppPathToManifests: webDriver.FindByXPath(`//table/tbody/tr/td[4]/div[text()="` + pathToManifests + `"]`)}
return &appPathElement
}

func GetMessageElements(webDriver *agouti.Page, msg string) *NamedElements {
successMsgElement := NamedElements{
HelmSuccessMessage: webDriver.FindByXPath(`//table//tbody/tr/td[text()="` + msg + ` install succeeded"]`),
KustomizeSuccessMessage: webDriver.FindByXPath(`//table/tbody/tr/td[text()="` + msg + `"]`)}

return &successMsgElement
}
28 changes: 20 additions & 8 deletions test/acceptance/test/pages/dashboard_page.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,27 @@ import (
"github.com/sclevine/agouti"
)

type DashboardWebDriver struct {
LogoImage *agouti.Selection
ApplicationTab *agouti.Selection
type DashboardPageElements struct {
LogoImage *agouti.Selection
ApplicationsHeader *agouti.Selection
AddAppButton *agouti.Selection
}

func Dashboard(webDriver *agouti.Page) DashboardWebDriver {
dashboard := DashboardWebDriver{
LogoImage: webDriver.FindByXPath(`//*[@id="app"]/div//img`),
ApplicationTab: webDriver.FindByXPath(`//*[@id="app"]//div/a/span/span[1]`)}
type AppListElements struct {
AppList *agouti.Selection
}

func GetDashboardPageElements(webDriver *agouti.Page) *DashboardPageElements {
dashboard := DashboardPageElements{
LogoImage: webDriver.FindByXPath(`//*[@id="app"]/div//img`),
ApplicationsHeader: webDriver.FindByXPath(`//*[@id="app"]//div/h2[text()="Applications"]`),
AddAppButton: webDriver.FindByXPath(`//*[@id="app"]//button[@span=(text()="Add Application")]`)}

return &dashboard
}

return dashboard
func GetAppListElements(webDriver *agouti.Page, appName string) *AppListElements {
appList := AppListElements{
AppList: webDriver.FindByXPath(`//*[@id="app"]//tbody/tr/td/span/a[contains(@href,'` + appName + `')]`)}
return &appList
}