Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tests: fmf/tmt combination for tests in upstrean and Fedora #1192

Merged
merged 3 commits into from
Apr 7, 2021
Merged

tests: fmf/tmt combination for tests in upstrean and Fedora #1192

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ecafb91
tests: fmf/tmt combination for tests in upstrean and Fedora
jscotka Apr 1, 2021
8b02a21
apply PR suggestions
jscotka Apr 7, 2021
dc715b6
missing sync of tests_recording important for fedora
jscotka Apr 7, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 5 additions & 1 deletion .packit.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
---
synced_files:
- src: fedora-tests/
- src: tests/
dest: tests/
- src: plans/
dest: plans/
- src: .fmf/
dest: .fmf/
# packit was already taken on PyPI
upstream_package_name: packitos
upstream_project_url: https://github.com/packit/packit
Expand Down
6 changes: 0 additions & 6 deletions fedora-tests/simple.py
View file
Open in desktop

This file was deleted.

26 changes: 0 additions & 26 deletions fedora-tests/tests.yml
View file
Open in desktop

This file was deleted.

25 changes: 0 additions & 25 deletions fedora-tests/upstream.sh
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion files/local-tests-requirements.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
project_dir: "{{ playbook_dir }}/.."
tasks:
- include_tasks: tasks/rpm-test-deps.yaml
- include_tasks: tasks/configure-git.yaml
- include_tasks: tasks/ogr.yaml
- include_tasks: tasks/sandcastle.yaml
# To (re-)generate tests_recording/test_data/
Expand Down
1 change: 0 additions & 1 deletion files/packit-testing-farm-prepare.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,4 @@
- include_tasks: tasks/generic-dnf-requirements.yaml
- include_tasks: tasks/python-compile-deps.yaml
- include_tasks: tasks/rpm-test-deps.yaml
- include_tasks: tasks/configure-git.yaml
- include_tasks: tasks/sandcastle.yaml
5 changes: 0 additions & 5 deletions files/tasks/configure-git.yaml
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion files/zuul-tests-session-recording.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
tasks:
- include_tasks: tasks/rpm-test-deps.yaml
- include_tasks: tasks/install-packit.yaml
- include_tasks: tasks/configure-git.yaml
- include_tasks: tasks/sandcastle.yaml
- name: Run session recorded tests
command: make check TESTS_TARGET=tests_recording
Expand Down
1 change: 0 additions & 1 deletion files/zuul-tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
tasks:
- include_tasks: tasks/rpm-test-deps.yaml
- include_tasks: tasks/install-packit.yaml
- include_tasks: tasks/configure-git.yaml
- include_tasks: tasks/sandcastle.yaml
- name: Run unit, integration and functional tests
command: make check
Expand Down
13 changes: 5 additions & 8 deletions plans/full.fmf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
summary:
Unit, integration & functional tests.
prepare:
how: ansible
playbooks:
- files/packit-testing-farm-prepare.yaml
summary: Unit, integration & functional tests.
discover:
how: fmf
filter: tier:1
execute:
script:
- make check
how: tmt
25 changes: 17 additions & 8 deletions plans/session-recording.fmf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,18 @@
summary:
Tests using pre-recorded responses from external services.
prepare:
how: ansible
playbooks:
- files/packit-testing-farm-prepare-session-recording.yaml
summary: Session recording testcases
discover:
how: fmf
filter: tier:2
execute:
script:
- make check TESTS_TARGET=tests_recording
how: tmt
adjust:
when: "distro <= rhel-8 or distro <= centos-8"
psss marked this conversation as resolved.
Show resolved Hide resolved
prepare:
- how: install
name: ensure_copr_plugin
order: 10
package:
- dnf-plugins-core
- how: shell
name: requre_copr
order: 20
script: 'dnf -y copr enable packit/packit-requre-main'
10 changes: 5 additions & 5 deletions plans/smoke.fmf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
summary:
Basic smoke tests.
summary: Basic simple tests
discover:
how: fmf
filter: tier:0
execute:
script:
- packit --version
- packit --help
how: tmt
34 changes: 34 additions & 0 deletions tests/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
import shutil
from pathlib import Path
from typing import Tuple
import subprocess
import tempfile

import pytest
from flexmock import flexmock
Expand All @@ -41,6 +43,21 @@
)


TMP_DIR = "/tmp/pytest_tmp_path/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be used only in the tmp_path fixture bellow, so could be made local to that function, no need to stay in the module scope.


# define own tmp_path fixture for older version of pytest (Centos)
try:
from _pytest import tmpdir

_ = tmpdir.tmp_path
except (ImportError, AttributeError, KeyError):

@pytest.fixture()
def tmp_path():
Path(TMP_DIR).mkdir(exist_ok=True, parents=True)
return Path(tempfile.mkdtemp(prefix=TMP_DIR))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I worry this won't work because tmp_path fixture provides unique paths for tests while this one is static for all - we'll see in the test results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea if fixture code is executed for every usage in def ...(tmp_path) But I hoped so :-)
so seems to work now somehow

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope="function", which means to create the fixture for each test function, is the default, but you could specify it to be more clear.



def get_git_repo_and_remote(
target_dir: Path, repo_template_path: Path
) -> Tuple[Path, Path]:
Expand Down Expand Up @@ -187,3 +204,20 @@ def copr_client_mock(get_list_return=None):

copr_mock = flexmock(mock_chroot_proxy=flexmock(get_list=lambda: get_list_return))
return copr_mock


@pytest.fixture(autouse=True)
def configure_git():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer this being a fixture set for the whole test session: https://docs.pytest.org/en/stable/fixture.html#autouse-fixtures-fixtures-you-don-t-have-to-request

CMDS = [
["git", "config", "--global", "user.email", "packit@redhat.com"],
["git", "config", "--global", "user.name", "Packit Test"],
]
# verify that git is already configured
try:
output = subprocess.check_output(["git", "config", "-l", "--global"])
except subprocess.CalledProcessError:
output = ""
if "user.name" in output if isinstance(output, str) else output.decode():
return
for item in CMDS:
subprocess.call(item)
24 changes: 24 additions & 0 deletions tests/full.fmf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
require:
- python3-flexmock
- python3-pytest
- python3-setuptools_scm_git_archive
- python3-setuptools_scm
- python3-distro
- python3-devel
- packit
- python3-packit
- python3-pytest-cov
- rpm-build
- rpmdevtools
- python3-bodhi-client
- fedpkg
- git
- make
#- rpmautospec-rpm-macros
component:
- packit
tier: 1
tag:
- basic
test: pytest-3 -v .
duration: 30m
23 changes: 21 additions & 2 deletions tests/integration/test_patches.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,21 @@
from pathlib import Path
from packit.patches import PatchGenerator, PatchMetadata

TESTS_DIR = str(Path(__file__).parent.parent)


def check_copytree_dirs_exists_support():
"""
Old version of shutil.copytree does not support
dirs_exist_ok parameter.
"""
if (
hasattr(shutil.copytree, "__defaults__")
and len(shutil.copytree.__defaults__) >= 5
):
Comment on lines +19 to +22
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be done pretty nicely with the inspect module: https://docs.python.org/3/library/inspect.html#inspect.getargspec

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, but I don't want to import inspect module there. And this seems to be sufficient. Especially as I've met more troubles with inspect module when using various version of Python.

return True
return False


@pytest.fixture
def git_repo(tmpdir):
Expand All @@ -18,20 +33,24 @@ def git_repo(tmpdir):
"""
repo = git.Repo.init(tmpdir)
shutil.copytree(
src="tests/data/patches/previous/",
src=f"{TESTS_DIR}/data/patches/previous/",
dst=repo.working_tree_dir,
dirs_exist_ok=True,
)
repo.git.add(repo.working_tree_dir)
repo.git.commit("-mInitial patches")
shutil.copytree(
src="tests/data/patches/regenerated/",
src=f"{TESTS_DIR}/data/patches/regenerated/",
dst=repo.working_tree_dir,
dirs_exist_ok=True,
)
return repo


@pytest.mark.skipif(
not check_copytree_dirs_exists_support(),
reason="Old python version does not support copytree exists dirs parameter",
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this #1160 ?

Copy link
Contributor Author

@jscotka jscotka Apr 7, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right it is exactly this issue, so it will be fixed/(test skipped) by this PR

def test_undo_identical(git_repo):
"""
Check that identical patches are correctly detected and changes
Expand Down
8 changes: 8 additions & 0 deletions tests/smoke.fmf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
require:
- packit
- python3-packit
summary: Basic smoke test
test: ./smoke.sh
tag:
- smoke
tier: 0
5 changes: 5 additions & 0 deletions tests/smoke.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash
set -eux

packit --version
packit --help
13 changes: 13 additions & 0 deletions tests_recording/main.fmf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
require:
- python3-flexmock
- python3-requre
- python3-pytest
- packit
- python3-packit
component:
- packit
tier: 2
tag:
- requre
test: pytest-3 -v .
duration: 10m