Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ProbabilitySamplingScheduler might be inverted #1881

Closed
DanBlackwell opened this issue Feb 24, 2024 · 3 comments · Fixed by #1884
Closed

ProbabilitySamplingScheduler might be inverted #1881

DanBlackwell opened this issue Feb 24, 2024 · 3 comments · Fixed by #1884
Assignees

Comments

@DanBlackwell
Copy link
Contributor

According to the comment for TestcaseScore::compute below, returning a higher value should make it more likely that a queue member is selected:

/// Computes the favor factor of a [`Testcase`]. Higher is better.

It looks to me like the reciprocal of this number is used in the ProbabilitySamplingScheduler, resulting in bigger compute scores ending up with lower probability of selection:

In my local version I have changed the above line to just let prob = factor, and it seems to behave more like I would expect. Perhaps I'm missing something though.

I'll also add that I am updating testcase scores, so I also added the following to the end of the function (if this is a new key then the subtraction never happens, so works for new insert and replace):

        if let Some(old_prob) = meta.map.insert(idx, prob) {
            // subtract the old probability from total first
            meta.total_probability -= old_prob;
        }
        meta.total_probability += prob;
@domenukk
Copy link
Member

@tokatoka can you take a look?

@tokatoka
Copy link
Member

yes i think you are right
can you send a PR?

@tokatoka
Copy link
Member

I'll also add that I am updating testcase scores, so I also added the following to the end of the function (if this is a new key then the subtraction never happens, so works for new insert and replace):

To implement this, can you impl RemovableScheduler for Probabilistic scheduler

DanBlackwell added a commit to DanBlackwell/LibAFL that referenced this issue Feb 26, 2024
tokatoka added a commit that referenced this issue Mar 5, 2024
* Fix for #1881, ProbabilitySamplingScheduler seems to be inverted

* Run 'cargo +nightly fmt'

* Added debug_assert as negative and infinite probability values would break the current implementation of next()

---------

Co-authored-by: Dan Blackwell <danblackwell@eduroam-int-dhcp-97-192-218.ucl.ac.uk>
Co-authored-by: Dan Blackwell <danblackwell@ML-W0NYVW4XCK.local>
Co-authored-by: Dongjia "toka" Zhang <tokazerkje@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants