[DRAFT] Remote test results #994
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is really intended to be the same for all instances of a given class. Every other (xunit, json, journal, html) result class already does it like that. Signed-off-by: Cleber Rosa <crosa@redhat.com>
Avocado defines avocado.core.result.TestResult, so that alternative test result classes can inherit from it. When a user writes a new test result class, he's left wondering how to activate that. First, for consistency, Avocado should define proper interfaces for "test result plugins". This is on sight, and will ultimately allow users to register new test result implementations simply by registering the setuptools entry points, just as it's currently done (on versions >= 0.31.0) for command line related plugins. Until then, let's at least improve the current state of things. These changes believe that the following improvements are given here: * avocado.core.job.Job._set_output_plugins now won't look at every possible key in the application arguments, and won't silence possible errors while trying to instantiate test result classes. * test result writers can now use a documented method for registering their test result classes. The same approach was not chosen to be done with the test runner classes because there can only by one test runner per Avocado job. Signed-off-by: Cleber Rosa <crosa@redhat.com>
This implementation of __getattr__ has allowed users of the test result proxy, most importantly the result test runner, to ask for attributes, including important methods, that would only be available on the RemoteTestResult implementation. Things have been working more or less by pure chance, based on the lucky fact that the RemoteTestResult would be at the right place on the proxy `output_plugins`. Let's forbid users of the test result proxy to look for attributes that are not part of the test result "contract" by removing this magical (and rather evil) method. Signed-off-by: Cleber Rosa <crosa@redhat.com>
And not the test proxy for attributes that are not part of the general test result "contract". NOTE: THIS IS A DRAFT CHANGE. IT INTRODUCES OTHER FAILURES WHEN RUNNING THE REMOTE BASED TEST RUNNER. It's sole purpose is to exemplify how the test result proxy `__getattr__` was being abused. Signed-off-by: Cleber Rosa <crosa@redhat.com>
|
I'm closing this one because the first two commits were sent on a separate PR, and the fix for the real issue is going to come on another PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a PR mostly addressed to @lmr since he reported the general issues with running custom test result implementations.
The first 2 commits are general improvements, but not critical changes. The last 2 commits really addresses the very broken state of remote test results and remote test runners. More complete description of the issues are on the commit message themselves.