Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a very basic smoke test. Run it with tox -e smoke #800

Merged
merged 6 commits into from Aug 10, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Expand Up @@ -11,3 +11,8 @@ venv
.vscode
.coverage
/.tox

# Smoke test artifacts
*.tar.gz
*.charm
test/charms/test_smoke/requirements.txt
9 changes: 9 additions & 0 deletions test/charms/test_smoke/README.md
@@ -0,0 +1,9 @@
# smoke

## Description

A simple test charm for running smoke tests.

## Usage

Run `tox -e smoke` in the root directory of this repository to build and deploy this charm.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you include the QA steps info from the PR description here - the stuff about how to make sure your environment is prepared to run the tests. Also probably worth noting that same info as a comment inside tox.ini.

22 changes: 22 additions & 0 deletions test/charms/test_smoke/charmcraft.yaml
@@ -0,0 +1,22 @@
# Learn more about charmcraft.yaml configuration at:
# https://juju.is/docs/sdk/charmcraft-config
type: "charm"
bases:
- build-on:
- name: "ubuntu"
channel: "20.04"
run-on:
- name: "ubuntu"
channel: "20.04"
- build-on:
- name: "ubuntu"
channel: "18.04"
run-on:
- name: "ubuntu"
channel: "18.04"
- build-on:
- name: "ubuntu"
channel: "16.04"
run-on:
- name: "ubuntu"
channel: "16.04"
10 changes: 10 additions & 0 deletions test/charms/test_smoke/config.yaml
@@ -0,0 +1,10 @@
# Copyright 2022 Penelope Valentine Gale
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the copyright not be Canonical?

# See LICENSE file for licensing details.
#
# TEMPLATE-TODO: change this example to suit your needs.
# If you don't need a config, you can remove the file entirely.
# It ties in to the example _on_config_changed handler in src/charm.py
#
# Learn more about config at: https://juju.is/docs/sdk/config

options: {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC we can remove the file completely if there's no config

12 changes: 12 additions & 0 deletions test/charms/test_smoke/metadata.yaml
@@ -0,0 +1,12 @@
# Copyright 2022 Penelope Valentine Gale
# See LICENSE file for licensing details.

Comment on lines +1 to +3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like maybe fixing this copyright line got missed before the merge?

# For a complete list of supported options, see:
# https://juju.is/docs/sdk/metadata-reference
name: smoke
display-name: |
smoke
description: |
smoke test charm
summary: |
basic minimal charm for running smoke tests
46 changes: 46 additions & 0 deletions test/charms/test_smoke/src/charm.py
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
#
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Charm the service.

Refer to the following post for a quick-start guide that will help you
develop a new k8s charm using the Operator Framework:

https://discourse.charmhub.io/t/4208
"""

import logging

from ops.charm import CharmBase
from ops.main import main
from ops.model import ActiveStatus

logger = logging.getLogger(__name__)


class SmokeCharm(CharmBase):
"""Charm the service."""

def __init__(self, *args):
super().__init__(*args)
self.framework.observe(self.on.install, self._on_install)

def _on_install(self, event):
self.unit.status = ActiveStatus()


if __name__ == "__main__":
main(SmokeCharm)
40 changes: 40 additions & 0 deletions test/smoke/test_smoke.py
@@ -0,0 +1,40 @@
# Copyright 2022 Canonical Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Learn more about testing at: https://juju.is/docs/sdk/testing

import logging

from pytest_operator.plugin import OpsTest

logger = logging.getLogger(__name__)


CURRENT_CHANNEL = '20.04' # Focal
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this used?



async def test_smoke(ops_test: OpsTest):
# Verify that we can deploy charms from supported series.

# Build the charm. (We just build it for focal -- it *should* work to deploy it on
# older versions of Juju.)
charm = await ops_test.build_charm("./test/charms/test_smoke/")

for series in ['focal', 'bionic', 'xenial']:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use pytest.mark.parametrize to give a better error message (use ids=['focal', ...] for bonus points)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started in on this, and it ended up adding a lot of complexity. The build needs to get moved into a separate routine, and then we need a data structure to track our charm artifact. None of that is difficult to write, but it didn't feel worthwhile to do so until this test gets more complex.

app = await ops_test.model.deploy(
charm, series=series, application_name="{}-smoke".format(series))
await ops_test.model.wait_for_idle(timeout=600)

assert app.status == "active", "Series {} failed with '{}' status".format(
series, app.status)
23 changes: 21 additions & 2 deletions tox.ini
Expand Up @@ -63,7 +63,7 @@ commands =

[testenv:unit]
description = Run unit tests
passenv =
passenv =
RUN_REAL_PEBBLE_TESTS
PEBBLE
deps =
Expand All @@ -74,7 +74,8 @@ deps =
coverage[toml]
-r{toxinidir}/requirements.txt
commands =
coverage run --source={[vars]src_path} -m pytest -v --tb native {posargs}
coverage run --source={[vars]src_path} \
-m pytest --ignore={[vars]tst_path}smoke -v --tb native {posargs}
coverage report

[testenv:pebble]
Expand All @@ -91,3 +92,21 @@ deps =
-r{toxinidir}/requirements.txt
commands =
bash -c "umask 0; (pebble run --http=':4000' --create-dirs &>/dev/null & ) ; sleep 1; pytest -v --tb native -k RealPebble {posargs} ; killall -y 3m pebble"

[testenv:smoke]
description = Run a smoke test against a Juju controller.
whitelist_externals = juju
charmcraft
bash
deps =
pytest
pytest-operator
commands =
# Build a source tarball for ops, and drop it into the root directory of the smoke test charm.
bash -c 'rm -vf ./test/charms/test_smoke/*.tar.gz # Cleanup old builds'
python {toxinidir}/setup.py sdist --dist-dir={toxinidir}/test/charms/test_smoke/
# Inject the tarball into the smoke test charm's requirements.
bash -c 'echo "./$(ls -1 ./test/charms/test_smoke/ | grep tar.gz)" > ./test/charms/test_smoke/requirements.txt'

# Run our smoke tests (this will build the charm, then run the tests).
pytest -v --tb native --log-cli-level=INFO -s {posargs} {toxinidir}/test/smoke/