-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RCORE-1928 Use baasaas for baas integration tests in CI #7423
Changes from all commits
42b5cfa
64737cb
5267fb2
74cec93
e41a531
c28d816
a5381d7
b8b2c24
56c7e43
845d7b2
af8424b
a6af90f
65fa191
4a65835
2c048cd
abd1c15
2c97874
c8b3b1f
970e236
b63cbde
7378d98
a54815e
95de793
4f4f406
208f565
2e96fc3
faae632
2dfe1db
1ece73b
0785694
7f55bcf
a0d375b
f7aeff0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -78,7 +78,6 @@ functions: | |
if [ -z "${disable_tests_against_baas|}" ]; then | ||
scheme="http" | ||
set_cmake_var baas_vars REALM_ENABLE_AUTH_TESTS BOOL On | ||
set_cmake_var baas_vars REALM_MONGODB_ENDPOINT STRING "$scheme://localhost:9090" | ||
if [ -n "${baas_admin_port|}" ]; then | ||
set_cmake_var baas_vars REALM_ADMIN_ENDPOINT STRING "$scheme://localhost:${baas_admin_port}" | ||
fi | ||
|
@@ -156,7 +155,7 @@ functions: | |
if [ -n "${curl_base|}" ]; then | ||
set_cmake_var curl_vars CURL_LIBRARY PATH "$(./evergreen/abspath.sh ${curl_base}/lib/libcurl.dll.a)" | ||
set_cmake_var curl_vars CURL_INCLUDE_DIR PATH "$(./evergreen/abspath.sh ${curl_base}/include)" | ||
set_cmake_var baas_vars REALM_CURL_CACERTS PATH "$(./evergreen/abspath.sh "${curl_base}/bin/cacert.pem")" | ||
set_cmake_var baas_vars REALM_CURL_CACERTS PATH "$(./evergreen/abspath.sh "${curl_base}/bin/curl-ca-bundle.crt")" | ||
fi | ||
|
||
set_cmake_var realm_vars REALM_NO_TESTS BOOL ${no_tests|Off} | ||
|
@@ -208,10 +207,17 @@ functions: | |
file: './realm-core/benchmark_results/results.latest.json' | ||
|
||
"run tests": | ||
- command: expansions.update | ||
params: | ||
file: realm-core/dependencies.yml | ||
- command: shell.exec | ||
params: | ||
working_dir: realm-core | ||
shell: bash | ||
env: | ||
BAASAAS_API_KEY: "${baasaas_api_key}" | ||
BAASAAS_REF_SPEC: "${BAAS_VERSION}" | ||
BAASAAS_START_MODE: "githash" | ||
script: |- | ||
set -o errexit | ||
set -o verbose | ||
|
@@ -244,6 +250,12 @@ functions: | |
|
||
TEST_FLAGS="--no-tests=error $TEST_FLAGS ${test_flags|}" | ||
|
||
if [[ -n "${disable_tests_against_baas|}" ]]; then | ||
unset BAASAAS_API_KEY | ||
unset BAASAAS_REF_SPEC | ||
unset BAASAAS_START_MODE | ||
fi | ||
|
||
if [[ -n "${llvm_symbolizer}" ]]; then | ||
export ASAN_SYMBOLIZER_PATH="$(./evergreen/abspath.sh ${llvm_symbolizer})" | ||
fi | ||
|
@@ -977,18 +989,17 @@ tasks: | |
test_label: objstore-local | ||
test_executable_name: "realm-object-store-tests" | ||
verbose_test_output: true | ||
disable_tests_against_baas: true | ||
- func: "check branch state" | ||
|
||
# These are baas object store tests that run against baas running on a remote host | ||
- name: baas-integration-tests | ||
tags: [ "test_suite", "for_pull_requests", "requires_baas" ] | ||
exec_timeout_secs: 3600 | ||
commands: | ||
- func: "launch remote baas" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be nice to eventually get rid of some of the launch/wait shell scripts, but I guess we'd have to wait to port over the network tests to this as well. Is there a viable path to doing this with baasaas? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, I was gonna do that in a separate PR. |
||
- func: "compile" | ||
vars: | ||
target_to_build: ObjectStoreTests | ||
- func: "wait for remote baas to start" | ||
- func: "run tests" | ||
vars: | ||
test_label: objstore-baas | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -145,39 +145,41 @@ These are the available variables: | |
testing process as soon as a check fails or an unexpected exception is thrown | ||
in a test. | ||
|
||
## Running [app] tests against a local MongoDB BAAS | ||
|
||
Due to MongoDB security policies, running baas requires company issued AWS account credentials. | ||
These are for MongoDB employees only, if you do not have these, reach out to #realm-core. | ||
Once you have them, they need to be set in the shell environment. | ||
|
||
First, log in to aws using their command line tool. On mac this requries `brew install awscli`. | ||
Then login using `aws configure` and input your access key and secret acess key. The other | ||
configuration options can be left as none. This creates a correctly formatted file locally at | ||
`~/.aws/credentials` which we will use later. | ||
|
||
If you do not want to install the aws command line tools, you can also create the aws file | ||
manually in the correct location (`~/.aws/credentials`) with the following contents: | ||
## Running [app] tests against a local BAAS instance | ||
|
||
If you already have a baas instance running, you can specify that directly via the | ||
`BAAS_BASE_URL` environment variable. You can run baas in a local docker container using | ||
instructions from [the wiki](https://wiki.corp.mongodb.com/display/10GEN/%28Device+Sync%29+Using+Docker+to+run+a+BAAS+server+instance). | ||
``` | ||
AWS_ACCESS_KEY_ID = <your-key-id> | ||
AWS_SECRET_ACCESS_KEY = <your-secret-key> | ||
export BAAS_BASE_URL=http://localhost:9090 | ||
mkdir build.sync.ninja | ||
cmake -B build.sync.ninja -G Ninja -DREALM_ENABLE_AUTH_TESTS=1 | ||
cmake --build build.sync.ninja --target realm-object-store-tests | ||
./build.sync.ninja/test/object-store/realm-object-store-tests -d=1 | ||
``` | ||
|
||
We use a script to fetch the dependencies for and run baas locally. Use the `-b sha` to use a particular version from https://github.com/10gen/baas/ | ||
The script uses the configuration from https://github.com/10gen/baas/blob/master/etc/configs/test_rcore_config.json | ||
## Running [app] tests against an on-demand BAASAAS container | ||
|
||
Due to MongoDB security policies, running baas requires company issued credentials. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it might be worth also pointing readers to https://wiki.corp.mongodb.com/display/10GEN/%28Device+Sync%29+Using+Docker+to+run+a+BAAS+server+instance and encouraging that workflow to be used for local development (the container is identical to what you get from launching one on baasaas). Baasaas should be reserved mostly for CI tasks. |
||
These are for MongoDB employees only, if you do not have these, reach out to | ||
#appx-device-sync-internal. Once you have a baasaas API key, it needs to be set | ||
in the shell environment. | ||
``` | ||
./evergreen/install_baas.sh -w baas | ||
``` | ||
|
||
To run the [app] tests against the local baas, you need to configure a build with some cmake options to tell the tests where to point to. | ||
``` | ||
export BAASAAS_API_KEY=<your API key here> | ||
mkdir build.sync.ninja | ||
cmake -B build.sync.ninja -G Ninja -DREALM_ENABLE_AUTH_TESTS=1 -DREALM_MONGODB_ENDPOINT=http://localhost:9090 | ||
cmake -B build.sync.ninja -G Ninja -DREALM_ENABLE_AUTH_TESTS=1 | ||
cmake --build build.sync.ninja --target realm-object-store-tests | ||
./build.sync.ninja/test/object-store/realm-object-store-tests -d=1 | ||
``` | ||
You can tell the object-store tests to use a specific version of baas with the | ||
`BAASAAS_START_MODE` environment variable, which can either be `githash`, `patchid`, | ||
or `branch`. If you specify a start mode, you need to tell it which githash or | ||
branch name to start with via the `BAASAAS_REF_SPEC` environment variable. Omitting | ||
these will use the latest available commit from the main branch of baas. | ||
|
||
If you've started a baasaas container already via the baasaas CLI, you can tell | ||
the object-store tests to use that with the `BAASAAS_INSTANCE_ID` environment variable. | ||
|
||
|
||
### Developing inside a container | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a bit confusing to have both the
disable_tests_against_baas
var andrequires_baas
. Can we have only one of these for clarity?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately no,
requires_baas
is a tag rather than an expansion which tells evergreen which tasks to run anddisable_tests_against_baas
is an expansion used in commands to tell them how to compile the object-store tests.