feat: Replace testcase in corpus when comparisons are closer

[plotchy]

This duplicates the CmpFeedback effort on the corpus.

The idea being that adding values to the corpus when the input reverts can still be useful to overcome comparisons.

ie: in this toy example, the magic value of 8650 is difficult to guess (without concolic).

pragma solidity ^0.8.13;
contract debug {
    function foo(uint x) public {
        if (x == uint(200000 / uint(23))) {
            assert(false);
        }
    }
}

However, if the input gets closer, the corpus will keep that input and mutate off that.

Concerns:

• The corpus will probably hugely over-inflate. I only tested this on the toy example.
• The map comparison takes a significant time. I went from ~62k exec/sec to ~50k exec/sec on this pr. This pr is dumb in the sense that it does all the comparisons once in Infant feedback and then again in Corpus feedback.

Pros:

• It works really well alongside the gaussian mutator. In debug target I went from finding the magic number in 6m22s to 29s with both

[fuzzland-bot]

Found: 19

Project Name	Vulnerability Found	Time Taken	Log
BIGFI_exp.txt	✅ Price Manipulation	0h-0m-51s	Log File
Shadowfi_exp.txt	❌	-1	Log File
SEAMAN_exp.txt	✅ Fund Loss	0h-1m-54s	Log File
BEGO_exp.txt	✅ Fund Loss	0h-0m-9s	Log File
cftoken_exp.txt	✅ Price Manipulation	0h-0m-21s	Log File
Carrot_exp.txt	❌	-1	Log File
MBC_ZZSH_exp.txt	❌	-1	Log File
AUR_exp.txt	❌	-1	Log File
SellToken_exp.txt	✅ Fund Loss	0h-0m-52s	Log File
ROI_exp.txt	✅ Fund Loss	0h-0m-22s	Log File
GPT_exp.txt	❌	-1	Log File
OLIFE_exp.txt	❌	-1	Log File
THB_exp.txt	✅ Fund Loss	0h-4m-11s	Log File
VerilogCTF.txt	❌‼️ Crashed	-1	Log File
CS_exp.txt	✅ Price Manipulation	0h-0m-16s	Log File
MintoFinance_exp.txt	✅ Fund Loss	0h-0m-11s	Log File
Novo_exp.txt	✅ Price Manipulation	0h-4m-17s	Log File
SELLC03_exp.txt	✅ Fund Loss	0h-2m-0s	Log File
Yyds_exp.txt	✅ Fund Loss	0h-0m-54s	Log File
DYNA_exp.txt	❌	-1	Log File
EAC_exp.txt	✅ Fund Loss	0h-0m-22s	Log File
Annex_exp.txt	❌	-1	Log File
PLTD_exp.txt	✅ Price Manipulation	0h-0m-45s	Log File
ApeDAO_exp.txt	✅ Price Manipulation	0h-0m-22s	Log File
GSS_exp.txt	✅ Fund Loss	0h-1m-10s	Log File
Axioma_exp.txt	✅ Fund Loss	0h-0m-35s	Log File
RFB_exp.txt	✅ Fund Loss	0h-0m-33s	Log File
HEALTH_exp.txt	✅ Price Manipulation	0h-0m-9s	Log File

[plotchy]

drafting this bc this isn't a good solution. it inflates the corpus too much and should have logic for corpus replacement on closer comparisons.

[plotchy]

After experimenting more I think doing this in should_replace() is probably better. No need for a Feedback

ityfuzz/src/fuzzer.rs

Lines 169 to 188 in 2ffa214

	/// Determine if a testcase should be replaced based on the minimizer map
	/// If the new testcase has a higher fav factor, replace the old one
	/// Returns None if the testcase should not be replaced
	/// Returns Some((hash, new_fav_factor, testcase_idx)) if the testcase
	/// should be replaced
	pub fn should_replace(&self, input: &I, coverage: &[u8; MAP_SIZE]) -> Option<(u64, f64, usize)> {
	let mut hasher = DefaultHasher::new();
	coverage.hash(&mut hasher);
	let hash = hasher.finish();
	// if the coverage is same
	if let Some((testcase_idx, fav_factor)) = self.minimizer_map.get(&hash) {
	let new_fav_factor = input.fav_factor();
	// if the new testcase has a higher fav factor, replace the old one
	if new_fav_factor > *fav_factor {
	return Some((hash, new_fav_factor, *testcase_idx));
	}
	}
	None
	}
	}

[plotchy]

I force pushed a new implementation. This adds a new metadata that is also checked in the should_replace function. It no longer inflates corpus. Now it replaces items with the same coverage map that have a better comparison map.

Here is a run with an extra trace that showed when a corpus was replaced
INFO Replacing testcase 1 ....

It solves the original example immediately whereas without this it takes some minutes.

================ EVM Fuzzer Start ===================


 INFO Deploying contract: test/debug.sol:debug
 INFO Contract test/debug.sol:debug deployed to: 0x4d4f5b9982db86a2fabcdc1ce0c4dbb734e578bb
 INFO Deployed all contracts

 INFO [Stats #0] run time: 0h-0m-0s, clients: 1, corpus: 0, objectives: 0, executions: 0, exec/sec: 0.000
 INFO ============= New Corpus Item =============
 INFO Reverted? false 
 Txn:
[Sender] 0x35c9dfd76bf02107ff4f7128Bd69716612d31dDb
   └─[1] 0x4D4f5b9982DB86A2fabcDc1cE0c4dbB734e578bB.foo(98112)

 INFO ==========================================
 INFO [Testcase #0] run time: 0h-0m-0s, clients: 1, corpus: 2, objectives: 0, executions: 1, exec/sec: 0.000
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO ============= Coverage Summary =============
 INFO test/debug.sol:debug(0x4d4f5b9982db86a2fabcdc1ce0c4dbb734e578bb): 60.24% Instruction Covered, 50.00% Branch Covered
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO Replacing testcase 1 with hash 17183980082107303669
 INFO [Stats #0] run time: 0h-0m-0s, clients: 1, corpus: 2, objectives: 0, executions: 1, exec/sec: 0.000
 INFO ============= New Corpus Item =============
 INFO Reverted? true 
 Txn:
[Sender] 0x35c9dfd76bf02107ff4f7128Bd69716612d31dDb
   ├─[1] 0x4D4f5b9982DB86A2fabcDc1cE0c4dbB734e578bB.foo(8695)
   │  └─ ← 0x4e487b710000000000000000000000000000000000000000000000000000000000000001

[fuzzland-bot]

Found: 18

Project Name	Vulnerability Found	Time Taken	Log
BIGFI_exp.txt	✅ Price Manipulation	0h-0m-40s	Log File
Shadowfi_exp.txt	❌	-1	Log File
SEAMAN_exp.txt	✅ Fund Loss	0h-1m-1s	Log File
BEGO_exp.txt	✅ Fund Loss	0h-2m-4s	Log File
cftoken_exp.txt	✅ Price Manipulation	0h-0m-32s	Log File
Carrot_exp.txt	✅ Arbitrary Call	0h-0m-17s	Log File
MBC_ZZSH_exp.txt	✅ Fund Loss	0h-3m-48s	Log File
AUR_exp.txt	❌	-1	Log File
SellToken_exp.txt	✅ Fund Loss	0h-0m-32s	Log File
ROI_exp.txt	✅ Fund Loss	0h-0m-11s	Log File
GPT_exp.txt	❌	-1	Log File
OLIFE_exp.txt	❌	-1	Log File
THB_exp.txt	❌	-1	Log File
VerilogCTF.txt	❌‼️ Crashed	-1	Log File
CS_exp.txt	✅ Price Manipulation	0h-0m-12s	Log File
MintoFinance_exp.txt	✅ Fund Loss	0h-0m-9s	Log File
Novo_exp.txt	❌	-1	Log File
SELLC03_exp.txt	✅ Fund Loss	0h-1m-30s	Log File
Yyds_exp.txt	✅ Fund Loss	0h-0m-45s	Log File
DYNA_exp.txt	❌	-1	Log File
EAC_exp.txt	❌	-1	Log File
Annex_exp.txt	❌	-1	Log File
PLTD_exp.txt	✅ Price Manipulation	0h-1m-3s	Log File
ApeDAO_exp.txt	✅ Price Manipulation	0h-0m-15s	Log File
GSS_exp.txt	✅ Fund Loss	0h-0m-46s	Log File
Axioma_exp.txt	✅ Fund Loss	0h-0m-17s	Log File
RFB_exp.txt	✅ Fund Loss	0h-0m-19s	Log File
HEALTH_exp.txt	✅ Price Manipulation	0h-0m-6s	Log File

[fuzzland-bot]

Found: 20

Project Name	Vulnerability Found	Time Taken	Log
BIGFI_exp.txt	✅ Price Manipulation	0h-1m-31s	Log File
Shadowfi_exp.txt	❌	-1	Log File
SEAMAN_exp.txt	✅ Fund Loss	0h-2m-1s	Log File
BEGO_exp.txt	✅ Fund Loss	0h-0m-17s	Log File
cftoken_exp.txt	✅ Price Manipulation	0h-0m-10s	Log File
Carrot_exp.txt	✅ Arbitrary Call	0h-0m-17s	Log File
MBC_ZZSH_exp.txt	✅ Fund Loss	0h-3m-24s	Log File
AUR_exp.txt	❌	-1	Log File
SellToken_exp.txt	✅ Fund Loss	0h-0m-20s	Log File
ROI_exp.txt	✅ Fund Loss	0h-0m-19s	Log File
GPT_exp.txt	❌	-1	Log File
OLIFE_exp.txt	❌	-1	Log File
THB_exp.txt	✅ Fund Loss	0h-3m-14s	Log File
VerilogCTF.txt	❌‼️ Crashed	-1	Log File
CS_exp.txt	✅ Price Manipulation	0h-0m-10s	Log File
MintoFinance_exp.txt	✅ Fund Loss	0h-0m-19s	Log File
Novo_exp.txt	✅ Price Manipulation	0h-1m-49s	Log File
SELLC03_exp.txt	✅ Fund Loss	0h-1m-15s	Log File
Yyds_exp.txt	✅ Fund Loss	0h-0m-42s	Log File
DYNA_exp.txt	❌	-1	Log File
EAC_exp.txt	❌	-1	Log File
Annex_exp.txt	❌	-1	Log File
PLTD_exp.txt	✅ Price Manipulation	0h-0m-41s	Log File
ApeDAO_exp.txt	✅ Price Manipulation	0h-0m-17s	Log File
GSS_exp.txt	✅ Fund Loss	0h-0m-32s	Log File
Axioma_exp.txt	✅ Fund Loss	0h-0m-37s	Log File
RFB_exp.txt	✅ Fund Loss	0h-2m-16s	Log File
HEALTH_exp.txt	✅ Price Manipulation	0h-0m-12s	Log File