[CX_HARDENING] - Develop A Better Framework for Input Permutation #3207
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ss-es
reviewed
May 22, 2024
ss-es
reviewed
May 23, 2024
jparr721
commented
May 30, 2024
| @@ -179,6 +179,15 @@ pub struct Expectations<TYPES: NodeType, S> { | |||
| pub task_state_asserts: Vec<Box<dyn Predicate<S>>>, | |||
| } | |||
|
|
|||
| impl<TYPES: NodeType, S> Expectations<TYPES, S> { | |||
| pub fn from_outputs(output_asserts: Vec<Box<dyn Predicate<Arc<HotShotEvent<TYPES>>>>>) -> Self { | |||
There was a problem hiding this comment.
@ss-es I added this because we only assert in two tests in the entire codebase, so it was easier than writing the same line over and over, and I don't think it hurts readability, though I could be swayed on a different name.
There was a problem hiding this comment.
Looks good to me! Honestly, you could probably even just call it outputs.
ss-es
previously approved these changes
May 31, 2024
ss-es
reviewed
May 31, 2024
ss-es
approved these changes
May 31, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #3161
This PR:
Inputs are mostly written for the happy path. The existing tests are written with a single stream of inputs, and this has bitten us in the past. However, as we see in the proposal_ordering tests, the permutation with index order is not sufficient and requires the implementor to arduously call a significant number of cases with manually prescribed indices. We can do better. We need a scalable and reliable way to permute inputs (and outputs) in a way that ensures that all possible orderings can be handled by the test.
This PR does just that, it adds a randomization layer over the inputs which can be specified by the new
randomandserialmacros. These macros make it easier to declare a vec with the context that you're intending for it to be used in without any additional thought.The new test script here will eventually completely replace the old one in
script.rsThis PR does not:
This PR does not rewrite every test, that would make the PR large and any changes would be painful. I have updated one test and will update the rest in a follow on PR
Key places to review: