Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a very basic smoke test. Run it with tox -e smoke #800

Merged
merged 6 commits into from Aug 10, 2022
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Expand Up @@ -11,3 +11,8 @@ venv
.vscode
.coverage
/.tox

# Smoke test artifacts
*.tar.gz
*.charm
test/charms/test_smoke/requirements.txt
11 changes: 11 additions & 0 deletions test/charms/test_smoke/README.md
@@ -0,0 +1,11 @@
# smoke

## Description

A simple test charm for running smoke tests.

## Usage

Make sure that you are on a box with charmcraft and juju installed, and that you are connected to a "machine" controller, such as a local lxd cloud.

Then, from the root directory of this repository, execute `tox -e smoke` to build and deploy this charm.
22 changes: 22 additions & 0 deletions test/charms/test_smoke/charmcraft.yaml
@@ -0,0 +1,22 @@
# Learn more about charmcraft.yaml configuration at:
# https://juju.is/docs/sdk/charmcraft-config
type: "charm"
bases:
- build-on:
- name: "ubuntu"
channel: "20.04"
run-on:
- name: "ubuntu"
channel: "20.04"
- build-on:
- name: "ubuntu"
channel: "18.04"
run-on:
- name: "ubuntu"
channel: "18.04"
- build-on:
- name: "ubuntu"
channel: "16.04"
run-on:
- name: "ubuntu"
channel: "16.04"
4 changes: 4 additions & 0 deletions test/charms/test_smoke/config.yaml
@@ -0,0 +1,4 @@
# Copyright 2022 Canonical
# See LICENSE file for licensing details.

options: {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC we can remove the file completely if there's no config

12 changes: 12 additions & 0 deletions test/charms/test_smoke/metadata.yaml
@@ -0,0 +1,12 @@
# Copyright 2022 Penelope Valentine Gale
# See LICENSE file for licensing details.

Comment on lines +1 to +3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like maybe fixing this copyright line got missed before the merge?

# For a complete list of supported options, see:
# https://juju.is/docs/sdk/metadata-reference
name: smoke
display-name: |
smoke
description: |
smoke test charm
summary: |
basic minimal charm for running smoke tests
46 changes: 46 additions & 0 deletions test/charms/test_smoke/src/charm.py
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
#
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Charm the service.

Refer to the following post for a quick-start guide that will help you
develop a new k8s charm using the Operator Framework:

https://discourse.charmhub.io/t/4208
"""

import logging

from ops.charm import CharmBase
from ops.main import main
from ops.model import ActiveStatus

logger = logging.getLogger(__name__)


class SmokeCharm(CharmBase):
"""Charm the service."""

def __init__(self, *args):
super().__init__(*args)
self.framework.observe(self.on.install, self._on_install)

def _on_install(self, event):
self.unit.status = ActiveStatus()


if __name__ == "__main__":
main(SmokeCharm)
37 changes: 37 additions & 0 deletions test/smoke/test_smoke.py
@@ -0,0 +1,37 @@
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Learn more about testing at: https://juju.is/docs/sdk/testing

import logging

from pytest_operator.plugin import OpsTest

logger = logging.getLogger(__name__)


async def test_smoke(ops_test: OpsTest):
# Verify that we can deploy charms from supported series.

# Build the charm. (We just build it for focal -- it *should* work to deploy it on
# older versions of Juju.)
charm = await ops_test.build_charm("./test/charms/test_smoke/")

for series in ['focal', 'bionic', 'xenial']:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use pytest.mark.parametrize to give a better error message (use ids=['focal', ...] for bonus points)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started in on this, and it ended up adding a lot of complexity. The build needs to get moved into a separate routine, and then we need a data structure to track our charm artifact. None of that is difficult to write, but it didn't feel worthwhile to do so until this test gets more complex.

app = await ops_test.model.deploy(
charm, series=series, application_name="{}-smoke".format(series))
await ops_test.model.wait_for_idle(timeout=600)

assert app.status == "active", "Series {} failed with '{}' status".format(
series, app.status)
23 changes: 21 additions & 2 deletions tox.ini
Expand Up @@ -63,7 +63,7 @@ commands =

[testenv:unit]
description = Run unit tests
passenv =
passenv =
RUN_REAL_PEBBLE_TESTS
PEBBLE
deps =
Expand All @@ -74,7 +74,8 @@ deps =
coverage[toml]
-r{toxinidir}/requirements.txt
commands =
coverage run --source={[vars]src_path} -m pytest -v --tb native {posargs}
coverage run --source={[vars]src_path} \
-m pytest --ignore={[vars]tst_path}smoke -v --tb native {posargs}
coverage report

[testenv:pebble]
Expand All @@ -91,3 +92,21 @@ deps =
-r{toxinidir}/requirements.txt
commands =
bash -c "umask 0; (pebble run --http=':4000' --create-dirs &>/dev/null & ) ; sleep 1; pytest -v --tb native -k RealPebble {posargs} ; killall -y 3m pebble"

[testenv:smoke]
description = Run a smoke test against a Juju controller.
whitelist_externals = juju
charmcraft
bash
deps =
pytest
pytest-operator
commands =
# Build a source tarball for ops, and drop it into the root directory of the smoke test charm.
bash -c 'rm -vf ./test/charms/test_smoke/*.tar.gz # Cleanup old builds'
python {toxinidir}/setup.py sdist --dist-dir={toxinidir}/test/charms/test_smoke/
# Inject the tarball into the smoke test charm's requirements.
bash -c 'echo "./$(ls -1 ./test/charms/test_smoke/ | grep tar.gz)" > ./test/charms/test_smoke/requirements.txt'

# Run our smoke tests (this will build the charm, then run the tests).
pytest -v --tb native --log-cli-level=INFO -s {posargs} {toxinidir}/test/smoke/