-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support multiple targets under test #67
Comments
cc @murali42 |
After running into this situation again, I think the originally proposed API is a bit too niche (it only allows controlling the config settings). With the addition of the Revised proposal In the example below, we have two targets under test, subject1 and subject2. Each has different config settings and aspects applied. This is done using a dict with some special keys. The dicts tell how to build the attribute. Within the analysis_test function, we mixin the extra aspects and cfg transition into those values to construct the
I'm not a fan of magic keys and raw dicts, but the two things we require are (1) the I suppose a builder-esq type of API would be possible, Another alternative would be to allow passing function for the attribute and push all the attribute construction onto the caller. e.g. Discarded ideas
|
This allows customizing what attributes are considered a target under test, i.e. that they have aspects, config settings, etc applied. This allows having multiple targets be tested, which is useful for comparing targets in different configurations, multiple targets with a custom configuration, or some combination of that. When this is used, the implementation function is passed a struct of the targets under test, keyed by their attribute name, instead of the singular target under test. Fixes #67 PiperOrigin-RevId: 588220718
This allows customizing what attributes are considered a target under test, i.e. that they have aspects, config settings, etc applied. This allows having multiple targets be tested, which is useful for comparing targets in different configurations, multiple targets with a custom configuration, or some combination of that. When this is used, the implementation function is passed a struct of the targets under test, keyed by their attribute name, instead of the singular target under test. Fixes #67 PiperOrigin-RevId: 588220718
After some discussion, going to revise it a bit:
|
This allows customizing what attributes are considered a target under test, i.e. that they have aspects, config settings, etc applied. This allows having multiple targets be tested, which is useful for comparing targets in different configurations, multiple targets with a custom configuration, or some combination of that. When this is used, the implementation function is passed a struct of the targets under test, keyed by their attribute name, instead of the singular target under test. Fixes #67 PiperOrigin-RevId: 588220718
This allows customizing what attributes are considered a target under test, i.e. that they have aspects, config settings, etc applied. This allows having multiple targets be tested, which is useful for comparing targets in different configurations, multiple targets with a custom configuration, or some combination of that. When this is used, the implementation function is passed a struct of the targets under test, keyed by their attribute name, instead of the singular target under test. Fixes #67 PiperOrigin-RevId: 588220718
This allows customizing what attributes are considered a target under test, i.e. that they have aspects, config settings, etc applied. This allows having multiple targets be tested, which is useful for comparing targets in different configurations, multiple targets with a custom configuration, or some combination of that. When this is used, the implementation function is passed a struct of the targets under test, keyed by their attribute name, instead of the singular target under test. Fixes #67 PiperOrigin-RevId: 588220718
This allows customizing what attributes are considered a target under test, i.e. that they have aspects, config settings, etc applied. This allows having multiple targets be tested, which is useful for comparing targets in different configurations, multiple targets with a custom configuration, or some combination of that. When this is used, the implementation function is passed a struct of the targets under test, keyed by their attribute name, instead of the singular target under test. Fixes #67 PiperOrigin-RevId: 588220718
This allows customizing what attributes are considered a target under test, i.e. that they have aspects, config settings, etc applied. This allows having multiple targets be tested, which is useful for comparing targets in different configurations, multiple targets with a custom configuration, or some combination of that. When this is used, the implementation function is passed a struct of the targets under test, keyed by their attribute name, instead of the singular target under test. Fixes #67 PiperOrigin-RevId: 588220718
On more than one occasion, I or others have needed to have multiple targets under test. This is usually done to verify that a particular configuration change doesn't change behavior from a base line, or to verify that the rule-under-test ignores some configuration change. By having a "baseline" with fixed config settings, it makes it possible to reliably check the expected vs actual state.
This is somewhat possible today, but is rather verbose. If config settings are involved, it's even more verbose, and if different config settings are involved for the targets under test, then even more.
Proposed API:
targets
arg. It is mutually exclusive with thetarget
arg.name -> <target under test kwargs>
.Hm, well, I'm not sure I 100% like this proposed api, but it works. I'm just not a fan of currying through arbitrary lists of kwargs through things.
The text was updated successfully, but these errors were encountered: