Feature/circleci rta #20769
Feature/circleci rta #20769
Conversation
dothebart
force-pushed
the
feature/circleci-rta
branch
2 times, most recently
from
207fc1b
to
763ea97
Compare
| @@ -42,7 +51,7 @@ parameters: | |||
| # these are basically global constants | |||
| build-targets: | |||
| type: string | |||
| default: "arangod arangoimport arangoexport arangodump arangorestore arangobench frontend" | |||
| default: "arangod arangoimport arangoexport arangodump arangorestore arangobench frontend arangovpack" | |||
There was a problem hiding this comment.
Why do we need arangovpack here?
There was a problem hiding this comment.
RTA expects to find all binaries that are there in distribution packages. arangovpack is part of that. We probably should have tests for it?
| @@ -233,6 +242,8 @@ jobs: | |||
| s3-prefix: | |||
| type: string | |||
| default: "" | |||
| arch: | |||
There was a problem hiding this comment.
Which are possible? What is default?
There was a problem hiding this comment.
this is generated by generate_config.py:482; it will depend on the current target architecture.
Its string is later on adjusted to download the go aux-binaries.
.circleci/generate_config.py
Outdated
| deployments = ['SG' | ||
| #"CL", | ||
| ] |
There was a problem hiding this comment.
should we run both by default or not? since its doubling the number of jobs. Basically a management decision.
| elif args.ui == "community": | ||
| add_rta_ui_test_jobs_to_workflow(args, workflow, build_config, build_job) | ||
| if args.ui == "only": |
There was a problem hiding this comment.
Why do we have logically inequal values within the same ui parameter?
There was a problem hiding this comment.
- we will allways run enterprise.
- if
communitywe will run those as well. - if
onlywe won't runtesting.jstests that would be added below.
There was a problem hiding this comment.
Questions/suggestions
- What is
testing.js? - what do you think about making
uian array, which can have any of these values:'enterprise' | 'community' | 'ARM' | 'testing.js'. So if we want to run both, we will set it to['enterprise', 'community'].
There was a problem hiding this comment.
testing.js is the framework which runs all the regular integration tests. Its using arangosh. Most of the time its probably not going to be of bigger interest for the person starting the UI tests - however the backend developers use them all the time.
The selenium tests are created under the python written RTA framework, which attempts to test installations and deployments - we use it during the release testing.
Regarding ARM, I don't think that running the tests on ARM yields different results, unless maybe we run them with instrumented binaries (which then tend to be slooouuwww)
Scope & Purpose
work on the ability to launch rta ui tests from circle-ci.
added Switches:
ui:offor not specified: disabled.only: run just the enterprise UI tests => Skipcommunity,ARM,testing.jstestscommunity: run the community UI tests as wellui-testsuites: coma separated list of UI testsuites to run (if not all). Cavehead: to parallize, this repo needs to know the list of tests as default.ui-deployments: corrosponds to--starter-modefrom RTA, atm only SG is specified by default.🍕 New feature