Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check test metadata sanity during discover step #1841

Open
psss opened this issue Feb 8, 2023 · 2 comments
Open

Check test metadata sanity during discover step #1841

psss opened this issue Feb 8, 2023 · 2 comments
Labels
discover Discover step good first issue Good for newcomers

Comments

@psss
Copy link
Collaborator

psss commented Feb 8, 2023

When the path key contains an invalid directory the problem is revealed only during the test execution (and can be quite hard to investigate). We should make sure that path exists for each tests identified during the discover step and warn user early.

@psss psss added good first issue Good for newcomers discover Discover step labels Feb 8, 2023
@idorax idorax self-assigned this Feb 9, 2023
@idorax
Copy link
Contributor

idorax commented Mar 15, 2023

Hi @psss, would you please paste the fmf file having invalid path? I'll give it a try and help to fix it.

Currently I failed to reproduce it, e.g.

$ tree tests
tests
└── BADPATH
    ├── main.fmf
    └── test.sh

2 directories, 2 files
$ cat tests/BADPATH/main.fmf
summary: Concise summary describing what the test does
path: /BADPATH
test: ./test.sh
framework: beakerlib
$ tmt -vvv run
/var/tmp/tmt/run-002
Found 1 plan.

/plans/foo
summary: Basic smoke test
    discover
        how: fmf
        order: 50
        directory: /home/huanli/dev/1841/foo
        summary: 1 test selected
            /tests/BADPATH
    provision
        how: local
        order: 50
        distro: Fedora Linux 36 (Thirty Six)
        kernel: 6.1.14-100.fc36.x86_64
        summary: 1 guest provisioned
    prepare
        how: install
        summary: Install required packages
        name: requires
        order: 70
        package: 1 package requested
            beakerlib
            cmd: /bin/bash -c 'rpm -q --whatprovides beakerlib || sudo dnf install -y beakerlib'
            out: beakerlib-1.29.3-1.fc36.noarch
    
        summary: 1 preparation applied
    execute
        how: tmt
        order: 50
        exit-first: False
            test: Concise summary describing what the test does
    finish
        prune: Prune plan workdir '/var/tmp/tmt/run-002/plans/foo'.
        summary: 0 tasks completed
Failed to write '/var/tmp/tmt/run-002/plans/foo/discover/default-0/tests/BADPATH/tmt-test-wrapper.sh'.
[Errno 2] No such file or directory: '/var/tmp/tmt/run-002/plans/foo/discover/default-0/tests/BADPATH/tmt-test-wrapper.sh'
$ 

@psss
Copy link
Collaborator Author

psss commented Aug 10, 2023

Actually, you have reproduced the issue well: During the test execution user gets:

Failed to write '/var/tmp/tmt/run-002/plans/foo/discover/default-0/tests/BADPATH/tmt-test-wrapper.sh'.
[Errno 2] No such file or directory: '/var/tmp/tmt/run-002/plans/foo/discover/default-0/tests/BADPATH/tmt-test-wrapper.sh'

Instead, it would be nice to give a clear warning during the discover steps that path: /BADPATH is invalid.

@idorax idorax removed their assignment May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discover Discover step good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants