Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integeration tests for mongodb_shard and mongodb_replicaset modules #53900

Open
wants to merge 9 commits into
base: devel
from

Conversation

Projects
None yet
3 participants
@rhysmeister
Copy link
Contributor

rhysmeister commented Mar 16, 2019

SUMMARY

Integration tests for MongoDB modules: mongodb_replicaset and mongodb_shard module tests included.

ISSUE TYPE
  • Feature Pull Request
ADDITIONAL INFORMATION

Tests can be executed as follows...

``
test/runner/ansible-test integration -v mongodb_replicaset --docker ubuntu1604
test/runner/ansible-test integration -v mongodb_shard --docker ubuntu1604

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Mar 16, 2019

The test ansible-test sanity --test shebang [explain] failed with 4 errors:

test/integration/targets/mongodb_replicaset/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_replicaset/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Mar 16, 2019

The test ansible-test sanity --test shebang [explain] failed with 4 errors:

test/integration/targets/mongodb_replicaset/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_replicaset/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Mar 16, 2019

The test ansible-test sanity --test shebang [explain] failed with 4 errors:

test/integration/targets/mongodb_replicaset/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_replicaset/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'

click here for bot help

@rhysmeister

This comment has been minimized.

Copy link
Contributor Author

rhysmeister commented Mar 16, 2019

@dagwieers Integration tests for MongoDB modules are here. There's a few Shippable tests not passing that I need some input on...

  1. Shebangs in bash scripts.

Not sure this is correct. All my shebangs are #!/bin/bash

https://app.shippable.com/github/ansible/ansible/runs/114252/1/console

01:06 Run command: /usr/bin/python test/sanity/code-smell/shebang.py
01:06 ERROR: Found 4 shebang issue(s) which need to be resolved:
01:06 ERROR: test/integration/targets/mongodb_replicaset/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash' (100%)
01:06 ERROR: test/integration/targets/mongodb_replicaset/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash' (100%)
01:06 ERROR: test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash' (100%)
01:06 ERROR: test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash' (100%)

  1. Couple of errors with a tasks file...

https://app.shippable.com/github/ansible/ansible/runs/114252/55/console

This is just a simple file task that makes sure a directory exists. I'm guessing this is a bug in the tests?

01:09 TASK [mongodb_replicaset : Ensure tests home exists] ***************************
01:09 fatal: [testhost]: FAILED! => {"changed": false, "msg": "Could not find coverage module."}

The same run also throws this error...

ERROR: Test target "mongodb_replicaset" has changed the test environment!

- name: Launch cfg server with auth
command: mongod --configsvr --port 4000 --dbpath /home/tests/config --logpath /home/tests/config.log --smallfiles --replSet "{{ configsrv_replicaset }}" --fork --auth --keyFile /root/my.key

- pause:

This comment has been minimized.

Copy link
@mattclay

mattclay Mar 18, 2019

Member

Integration tests should not pause. This integration test has over 4 minutes of pauses in it.

This comment has been minimized.

Copy link
@rhysmeister

rhysmeister Mar 19, 2019

Author Contributor

@mattclay They need to pause to allow processes to become active and elections to happen. The alternative would be to continually poll the replicaset status and we effectively still pause anyway.

This comment has been minimized.

Copy link
@mattclay

mattclay Mar 20, 2019

Member

Polling would be ideal, perhaps with wait_for if possible. That way the tests can continue as soon as the services are ready instead of waiting the full 10 seconds. Getting the test run time down is important, and polling will hopefully make a significant difference.

This comment has been minimized.

Copy link
@rhysmeister

rhysmeister Mar 22, 2019

Author Contributor

Polling has been replaced in the majority of instances. I use a js script to poll the mongod instance and return once it's in the primary status. Seem to work well.

Show resolved Hide resolved test/integration/targets/mongodb_replicaset/aliases
state: directory

- name: Execute cleanup script if /home/tests/mongodb[1-3] directories exist
script: files/bash/mongod_teardown.sh

This comment has been minimized.

Copy link
@mattclay

mattclay Mar 18, 2019

Member

Instead of invoking shell scripts, include a file with the necessary tasks.

This comment has been minimized.

Copy link
@rhysmeister

rhysmeister Mar 23, 2019

Author Contributor

Bash scripts have been replaced with a bunch of include_tasks and other bits. Not really on board with the point of this as I think it's one of the rare occasions where bash scripts are a more elegant solution than Ansible tasks.

Can you have a look and comment if I'm on the right track. I've made these changes for the mongodb_replicaset tests only. mongodb_shard tests will be a little more complicated since there's multiple replicasets, config server and so on.

This comment has been minimized.

Copy link
@mattclay

mattclay Apr 10, 2019

Member

Yes, that's much better.

Using include tasks will make it easier to use remote_tmp_dir instead of /home/tests as well.

Show resolved Hide resolved test/integration/targets/setup_mongodb/defaults/main.yml Outdated
Show resolved Hide resolved test/integration/targets/setup_mongodb/defaults/main.yml Outdated
Show resolved Hide resolved test/integration/targets/mongodb_shard/tasks/main.yml Outdated
destructive
shippable/posix/group1
skip/osx
skip/freebsd

This comment has been minimized.

Copy link
@mattclay

mattclay Mar 18, 2019

Member

Why are the tests being skipped on macOS and FreeBSD?

This comment has been minimized.

Copy link
@rhysmeister

rhysmeister Mar 19, 2019

Author Contributor

Wanted to keep things simpler for the moment. Setting up a MongoDB cluster on . FreeBSD is a fair bit different to Linux and I'm not entirely sure a port of mongos is available for sharded clusters..

@ansibot ansibot removed the ci_verified label Mar 19, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Mar 19, 2019

The test ansible-test sanity --test shebang [explain] failed with 4 errors:

test/integration/targets/mongodb_replicaset/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_replicaset/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Mar 20, 2019

The test ansible-test sanity --test shebang [explain] failed with 4 errors:

test/integration/targets/mongodb_replicaset/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_replicaset/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'

click here for bot help

@ansibot ansibot removed the ci_verified label Mar 22, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Mar 22, 2019

The test ansible-test sanity --test shebang [explain] failed with 4 errors:

test/integration/targets/mongodb_replicaset/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_replicaset/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Mar 23, 2019

The test ansible-test sanity --test shebang [explain] failed with 2 errors:

test/integration/targets/mongodb_shard/files/bash/mongod_teardown.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'
test/integration/targets/mongodb_shard/files/bash/mongodb_replicaset.sh:1:1: unexpected non-module shebang: b'#!/bin/bash'

click here for bot help

@ansibot ansibot added the ci_verified label Mar 23, 2019

@ansibot ansibot added the stale_ci label Mar 31, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.