Replace sharness-based test suite by pytest #1401
Conversation
There was a problem hiding this comment.
Instead of replicating the skipif marks everywhere, they could be defined once in a helper module as suggested here: https://docs.pytest.org/en/8.2.x/how-to/skipping.html#id1
You should probably run https://github.com/astral-sh/ruff on the new code. I'd suggest a .ruff.toml with line-length = 119 (the same our other projects).
ejoerns
force-pushed
the
pytest
branch
2 times, most recently
from
ad34ec1
to
3d9a721
Compare
|
@jluebbe Thank you for the quick review!
Indeed. While the skipif statements were easy to copy&paste, they are probably not that easy to maintain and change and pollute the set files a bit. Fixed.
Ran Also the other review comments should be addressed by this updated version. |
|
As we don't have pytest in our container, this the tests are not yet running in CI. We should add the necessary dependencies in a separate PR first. |
I've created #1410 for this. I have updated this PR to contain a commit referencing the Docker container built by #1410. I've also renamed all test files and test cases to not contain Addressed the remaining review comments. |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
I've updated the PR and rebased to drop the intermediate container commit and add a workaround for 9pfs-related issues using sendfile() for file copying in qemu-test as suggested by @jluebbe |
|
Added some missing bundle copying in convert tests that previously failed for cross (386) only (for some reason). |
|
Applied two minor cleanups |
|
Force-pushed to sort imports and to remove some redundant comments. |
helper.py partly copied from rauc-hawkbit-updater. Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Only if pytest is available. Since the tests require a notable amount of disk space, we need to limit the number of temporary directories kept, according to pytest doc, using --basetemp is a proper workaround: > The number of entries currently cannot be changed, but using the > --basetemp option will remove the directory before every run, > effectively meaning the temporary directories of only the most recent > run will be kept. [1] https://docs.pytest.org/en/7.1.x/how-to/tmp_path.html#the-default-base-temporary-directory Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
Signed-off-by: Enrico Joerns <ejo@pengutronix.de>
In the past, we faced many limitations of the sharness-based RAUC test suite and additionally, the growing size of the
rauc.tfile made it harder to keep oversight over the test cases.With the sharness-based tests, we were often limited to having a simple program exit code check. But the reasons for exiting in RAUC are manifold.
With pytest, we can easily check for expected stdout/stderr output. Almost all tests were extended with this simple enhancement.
The new tests are now also split up into different test files, based on their test purpose.