Does `cargo afl build` also instrument all the dependency crates?

[ihayet]

Suppose I have a top-level project A which has dependencies on crates B, C, and D. If I use cargo afl build from the top-level project, will only A be instrumented for coverage or will all of B, C, and D be instrumented for coverage as well? If yes, is it possible to introduce a flag to discard the dependencies from being instrumented?

[smoelius]

All four dependencies should be intrumented.

If yes, is it possible to introduce a flag to discard the dependencies from being instrumented?

I can't see an easy way to do this. cargo-afl uses RUSTFLAGS to compile a package with instrumentation:

afl.rs/cargo-afl/src/main.rs

Line 322 in 902cf32

rustflags.push_str(&format!(

So I think what you're asking for would require B, C, and D to be compiled with different RUSTFLAGS than A, and I don't think rustc supports this today. (See this comment from dtolnay, for example.)

Can I ask why you want this feature?

[ihayet]

I understand. Thanks for the reply. I am asking to check if instrumenting all dependencies might increase the search space and introduce noise in the coverage guidance process? If not then I think it should be fine if the dependencies are also instrumented.

I found some success by compiling my dependency as a static library. Later I built my top-level project with cargo afl build and manually linked the static library to my top-level project. Then I used a C ABI to call one of the library functions. I validated that in this way only the top-level project was instrumented. I can imagine doing this for one or two dependencies with a manageable number of functions. However, this might become infeasible for many dependencies with many functions. Can you please suggest any alternative and scalable approach?

[smoelius]

I understand. Thanks for the reply. I am asking to check if instrumenting all dependencies might increase the search space and introduce noise in the coverage guidance process? If not then I think it should be fine if the dependencies are also instrumented.

Is it possible to make that question more concrete?

If A calls into B, C, and D, my intuition says you would want to know the paths are taken through B, C, and D. Can you give an example where you wouldn't?

I found some success by compiling my dependency as a static library. Later I built my top-level project with cargo afl build and manually linked the static library to my top-level project. Then I used a C ABI to call one of the library functions. I validated that in this way only the top-level project was instrumented. I can imagine doing this for one or two dependencies with a manageable number of functions. However, this might become infeasible for many dependencies with many functions. Can you please suggest any alternative and scalable approach?

I cannot immediately think of a better approach.

I am impressed, though. What you did sounds like quite a feat!

[ihayet]

Is it possible to make that question more concrete?

Yes, the rationale was that if the search space is smaller, would it be possible to focus entirely on A and cover more of A faster? At the same time, it might be possible that only a fraction of B, C, and D's code might be used by A. As a result, the majority of B, C, and D might not be covered at all. But, I agree with you that knowing if paths are taken through B, C, and D should also be useful.

One simpler alternative that I tried was to run the command cargo build -vv and I retrieved all the rustc compilation commands for the dependencies B, C, and D. Then I ran cargo afl build -vv and I retrieved the rustc compilation command only for the top-level project A. Then I ran the retrieved commands in order. The build was successful and afl-showmap somewhat validated that much fewer edges were being covered which might mean that only the top-level project A was instrumented. But, I think more validation is required for this approach.

[smoelius]

Yes, the rationale was that if the search space is smaller, would it be possible to focus entirely on A and cover more of A faster? At the same time, it might be possible that only a fraction of B, C, and D's code might be used by A. ...

What I am struggling with is how ignoring paths through B, C, and D would lead to finding bugs in A faster.

I presume A, ..., D are private and cannot be shared. Is there a way to give analogs that would make the situation clearer?

[ihayet]

I don't have a specific concrete example since I was hoping to do some fuzzing exercises on some popular crates (e.g. quick-xml). I wanted to make sure if it would be fine to have the dependencies instrumented as well.

I was considering a hypothetical situation. Suppose A calls a function B::foo(arg1, arg2) where different values of arg1 and arg2 might trigger different paths (e.g., path1, path2, path3) inside B. If a certain seed triggers B->path1, the seed is added to the fuzzing queue if we use coverage guidance from B. In this way, we keep adding to the fuzzing queue - interesting seeds that trigger new paths only inside B. From an overall bug-finding perspective, it is certainly interesting to exercise new paths in B. But if the fuzzing queue is influenced more by B, then will that slow down the coverage of A?

[smoelius]

From an overall bug-finding perspective, it is certainly interesting to exercise new paths in B. But if the fuzzing queue is influenced more by B, then will that slow down the coverage of A?

OK. I think I see what you mean.

Like suppose there were two paths through A, one that led to B and one that didn't. And suppose we had good reason to believe the bugs in A were along the path that didn't lead to B. Could instrumenting B cause us to spend too much time on the wrong path through A?

Yeah, that sounds plausible. But, unfortunately, there's no way to do exactly what you've asked, at least not currently.

Other ways of achieving a similar effect include (and I'm sure there are others):

• Craft the inputs to A in a way that you know will not go down the path to B.
• Detect inputs that would lead to B early and reject them.

To be sure I understand, your approach to not instrumenting B, C, and D involved building the final binary with rustc directly, and not cargo, correct?

[ihayet]

Other ways of achieving a similar effect include (and I'm sure there are others):

• Craft the inputs to A in a way that you know will not go down the path to B.
• Detect inputs that would lead to B early and reject them.

Thanks for the suggestions. I will keep this in mind.

To be sure I understand, your approach to not instrumenting B, C, and D involved building the final binary with rustc directly, and not cargo, correct?

Yes, my second approach involved building the final binary with rustc directly.

[smoelius]

Just for the record, I am not opposed to adding a feature to cargo-afl to not instrument certain packages. But cargo-afl is tightly coupled to cargo, and I cannot see a way to make this work without decoupling them. If such a way existed, I would consider it.

[ihayet]

Yes, I agree. Selective instrumentation is indeed challenging with a tight coupling between cargo-afl and cargo.

[smoelius]

Dropping this here just for reference: https://github.com/AFLplusplus/AFLplusplus/blob/stable/instrumentation/README.instrument_list.md

[ihayet]

Thanks for sharing. I will give this a try.

[smoelius]

Sorry, I should have been more clear.

I happened upon that AFL++ feature, but it's not immediately clear to me that it would work for Rust files.

Of course, if you could get it to work, that would be interesting.

[vanhauser-thc]

it does work but requires nightly and building the afl++ llvm plugins

[smoelius]

it does work but requires nightly and building the afl++ llvm plugins

Thanks, @vanhauser-thc. Can I ask why the feature requires plugins?