Penalize testcase that has found objectives #2093
Conversation
| tc.set_objectives_found(true); | ||
| } | ||
| Err(_) => { | ||
| // just do nothing, we are about to die. returning error makes no sense |
There was a problem hiding this comment.
ok.
but i have the fundamental question. is it safe to access current_testcase_mut() here?
| @@ -339,6 +343,10 @@ where | |||
| weight *= 2.0; | |||
| } | |||
|
|
|||
| if entry.objectives_found() { | |||
There was a problem hiding this comment.
objectives_found sounds like a counter (should it be?)
There was a problem hiding this comment.
no. it's boolean.
then found_objectives?
There was a problem hiding this comment.
Renaming is fine, too. But counting doesn't cost performance and maybe it's helpful for some other algorithms?
|
BTW., I feel like crashes are close to other crashes, so scheduling this guy less often may lead to less objectives? But I understand it makes the overall fuzzing faster, so... fuzzbench? |
In reality, the "other crashes" are just the same crashes but hit once edges one time more than the previous one |
|
btw fuzzbench won't give you an bug coverage stuff. (i mean we know the # of objectives but it is not the "coverage") |
That may be right, but it's still only gut feeling.. |
|
(And gut feeling is always wrong in fuzzing :P) |
|
Not very effective |
|
I would still add the information to the testcase, though? It can be handy for sure |
libafl/src/corpus/testcase.rs
Outdated
| @@ -65,6 +65,8 @@ where | |||
| parent_id: Option<CorpusId>, | |||
| /// If the testcase is "disabled" | |||
| disabled: bool, | |||
| /// has found crash (or timeout) or not | |||
| found_objectives: bool, | |||
There was a problem hiding this comment.
Like, let's make this a counter, and if noone uses it, fine, but if they do, even better
also, didn't this guy win?
|
|
@tokatoka how about this? |
|
We could even just do a smaller penalty? |
|
Added a small penalty @tokatoka |
the overall score, yes, but it is just because one benchmark didn't build for libafl. keeping the data is fine, but i wouldn't change the score. (because i don't want this implementation diverge from AFL++ |
|
|
||
| /// Adds one objectives to the `objectives_found` counter. Mostly called from crash handler or executor. | ||
| pub fn found_objective(&mut self) { | ||
| let _ = self.objectives_found.saturating_add(1); |
There was a problem hiding this comment.
This doesn't do anything.
https://doc.rust-lang.org/std/primitive.usize.html#method.saturating_add
There was a problem hiding this comment.
thank you for pointing this out! 👍
No description provided.