From e81ecf7be8f138b3b2dbe394f7642a92a24f4468 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Thu, 23 Mar 2023 14:29:46 +0100 Subject: [PATCH 1/8] Added discussion on how and when to use cheat codes --- .../echidna/advanced/on-using-cheat-codes.md | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 program-analysis/echidna/advanced/on-using-cheat-codes.md diff --git a/program-analysis/echidna/advanced/on-using-cheat-codes.md b/program-analysis/echidna/advanced/on-using-cheat-codes.md new file mode 100644 index 00000000..19e4b3d8 --- /dev/null +++ b/program-analysis/echidna/advanced/on-using-cheat-codes.md @@ -0,0 +1,53 @@ +# How and when to use cheat codes + +**Table of contents:** + +- [How and when to use cheat codes](#how-and-when-to-use-cheat-codes) + - [Introduction](#introduction) + - [Cheat codes available in Echidna](#cheat-codes-available-in-echidna) + - [Advise on using cheat codes](#advise-on-using-cheat-codes) + +## Introduction + +When solidity smart contract testing is performed from Solidity itself, usually requires some "help" in order to tackle some EVM/Solidity limitations. +Cheat code are special functions that allow to change the state of the EVM in ways that are not posible in production. These were introduced by Dapptools in +hevm and adopted (and expanded) in other projects such as Foundry. + +## Cheat codes available in Echidna + +Since Echidna uses [hevm](https://github.com/ethereum/hevm), all the supported list of cheat code is documented here: https://hevm.dev/controlling-the-unit-testing-environment.html#cheat-codes. +If a new cheat code is added in the future, Echidna only needs to update the hevm version and everything should work out of the box. + +As an example, this is code "simulates" the use of another sender for the external call using "prank": + +```solidity +interface IHevm { + function prank(address) external; +} + +contract TestPrank { + address constant HEVM_ADDRESS = 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D; + IHevm hevm = IHevm(HEVM_ADDRESS); + Contract c = ... + + function prankContract() public payable { + hevm.prank(address(0x42424242); + c.f(); + } +} +``` + +A specific example on the use of `sign` cheat code is available [here in our documentation](hevm-cheats-to-test-permit.md). + +## Advise on how and when using cheat codes + +While we provide support for the use of cheat codes, these should be used responsabily. We offer the following advise on the use of cheat codes: + +* It should be used only if Echidna will not perform the same action with a native feature. For instance, Echidna automatically increases the timestamp and block number. There are [some reports of the optimizer interfering with (re)computation of the block.number or timestamp](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which could generate incorrect tests when using cheat codes. +Using the corresponding built-in Echidna features should never intefer with the optimization level or any other compiler feature (if this happens, then it is a bug). + +* It can introduce false positives on the testing. For instance, using `prank` to simulate calls from account that is not EOA (e.g. a contract) can allow transactions that are not possible in the blockchain. + +* Using too much cheat codes: + * can be confusing or error-prone. Certain cheat code like `prank` allow to change caller in the next external call: It can be difficult to follow, in particular if it is used in internal functions or modifiers. + * will create a dependency of your code with the particular tool or cheat code implementation: It can cause produce migrations to other tools or reusing the test code to be more difficult than expected. From f00289fc0609c374a6f86cdfdbd08c1b0fe3201d Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Thu, 23 Mar 2023 14:31:49 +0100 Subject: [PATCH 2/8] Update README.md --- program-analysis/echidna/advanced/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/program-analysis/echidna/advanced/README.md b/program-analysis/echidna/advanced/README.md index 4642f151..8ae718a0 100644 --- a/program-analysis/echidna/advanced/README.md +++ b/program-analysis/echidna/advanced/README.md @@ -6,6 +6,7 @@ - [How to perform smart contract fuzzing at a large scale](./smart-contract-fuzzing-at-scale.md): How to use Echidna to run a long fuzzing campaign for complex smart contracts. - [How to test a library](https://blog.trailofbits.com/2020/08/17/using-echidna-to-test-a-smart-contract-library/): How Echidna was used to test the library in Set Protocol (blogpost) - [How to test bytecode-only contracts](./testing-bytecode.md): How to fuzz a contract without bytecode or to perform differential fuzzing between Solidity and Vyper +- [How and when to use cheat codes](./on-using-cheat-codes.md): How to use hevm cheat codes in general - [How to use hevm cheats to test permit](./hevm-cheats-to-test-permit.md): How to test code that depends on ecrecover signatures using hevm cheat codes - [How to seed Echidna with unit tests](./end-to-end-testing.md): How to use existing unit tests to seed Echidna - [Understanding and using `multi-abi`](./using-multi-abi.md): What is `multi-abi` testing, and how can it be used From ff20c0e7c87440ed0b3771bcad0467cd83170366 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Fri, 24 Mar 2023 15:22:17 +0100 Subject: [PATCH 3/8] Update on-using-cheat-codes.md --- .../echidna/advanced/on-using-cheat-codes.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/program-analysis/echidna/advanced/on-using-cheat-codes.md b/program-analysis/echidna/advanced/on-using-cheat-codes.md index 19e4b3d8..36845e54 100644 --- a/program-analysis/echidna/advanced/on-using-cheat-codes.md +++ b/program-analysis/echidna/advanced/on-using-cheat-codes.md @@ -5,20 +5,19 @@ - [How and when to use cheat codes](#how-and-when-to-use-cheat-codes) - [Introduction](#introduction) - [Cheat codes available in Echidna](#cheat-codes-available-in-echidna) - - [Advise on using cheat codes](#advise-on-using-cheat-codes) + - [Advice on using cheat codes](#advice-on-using-cheat-codes) ## Introduction -When solidity smart contract testing is performed from Solidity itself, usually requires some "help" in order to tackle some EVM/Solidity limitations. -Cheat code are special functions that allow to change the state of the EVM in ways that are not posible in production. These were introduced by Dapptools in -hevm and adopted (and expanded) in other projects such as Foundry. +When testing smart contracts in Solidity itself, it can be helpful to use cheat codes in order to overcome some of the limitations of the EVM/Solidity. +Cheat codes are special functions that allow to change the state of the EVM in ways that are not posible in production. These were introduced by Dapptools in hevm and adopted (and expanded) in other projects such as Foundry. ## Cheat codes available in Echidna Since Echidna uses [hevm](https://github.com/ethereum/hevm), all the supported list of cheat code is documented here: https://hevm.dev/controlling-the-unit-testing-environment.html#cheat-codes. If a new cheat code is added in the future, Echidna only needs to update the hevm version and everything should work out of the box. -As an example, this is code "simulates" the use of another sender for the external call using "prank": +As an example, the `prank` cheat code is able to set the `msg.sender` address in the context of the next external call: ```solidity interface IHevm { @@ -32,16 +31,16 @@ contract TestPrank { function prankContract() public payable { hevm.prank(address(0x42424242); - c.f(); + c.f(); // `c` will be called with `msg.sender = 0x42424242` } } ``` A specific example on the use of `sign` cheat code is available [here in our documentation](hevm-cheats-to-test-permit.md). -## Advise on how and when using cheat codes +## Advice on how and when using cheat codes -While we provide support for the use of cheat codes, these should be used responsabily. We offer the following advise on the use of cheat codes: +While we provide support for the use of cheat codes, these should be used responsibly. We offer the following advice on the use of cheat codes: * It should be used only if Echidna will not perform the same action with a native feature. For instance, Echidna automatically increases the timestamp and block number. There are [some reports of the optimizer interfering with (re)computation of the block.number or timestamp](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which could generate incorrect tests when using cheat codes. Using the corresponding built-in Echidna features should never intefer with the optimization level or any other compiler feature (if this happens, then it is a bug). From 6f1ee24522a47d13987db1356dd66095370f9b8b Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Fri, 24 Mar 2023 15:40:42 +0100 Subject: [PATCH 4/8] Update on-using-cheat-codes.md --- .../echidna/advanced/on-using-cheat-codes.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/program-analysis/echidna/advanced/on-using-cheat-codes.md b/program-analysis/echidna/advanced/on-using-cheat-codes.md index 36845e54..c4f68d93 100644 --- a/program-analysis/echidna/advanced/on-using-cheat-codes.md +++ b/program-analysis/echidna/advanced/on-using-cheat-codes.md @@ -13,9 +13,8 @@ When testing smart contracts in Solidity itself, it can be helpful to use cheat Cheat codes are special functions that allow to change the state of the EVM in ways that are not posible in production. These were introduced by Dapptools in hevm and adopted (and expanded) in other projects such as Foundry. ## Cheat codes available in Echidna - -Since Echidna uses [hevm](https://github.com/ethereum/hevm), all the supported list of cheat code is documented here: https://hevm.dev/controlling-the-unit-testing-environment.html#cheat-codes. -If a new cheat code is added in the future, Echidna only needs to update the hevm version and everything should work out of the box. +Echidna supports all cheat codes that are available in [hevm](https://github.com/ethereum/hevm). These are documented here: https://hevm.dev/controlling-the-unit-testing-environment.html#cheat-codes. +If a new cheat code is added in the future, Echidna only needs to update the hevm version and everything should work out of the box. As an example, the `prank` cheat code is able to set the `msg.sender` address in the context of the next external call: @@ -42,11 +41,10 @@ A specific example on the use of `sign` cheat code is available [here in our doc While we provide support for the use of cheat codes, these should be used responsibly. We offer the following advice on the use of cheat codes: -* It should be used only if Echidna will not perform the same action with a native feature. For instance, Echidna automatically increases the timestamp and block number. There are [some reports of the optimizer interfering with (re)computation of the block.number or timestamp](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which could generate incorrect tests when using cheat codes. -Using the corresponding built-in Echidna features should never intefer with the optimization level or any other compiler feature (if this happens, then it is a bug). +* It can break certain assumptions in Solidity. For example, the compiler assumes that `block.number` is constant during a transaction. There are [reports of the optimizer interfering with (re)computation of the `block.number` or `block.timestamp`](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which can generate incorrect tests when using cheat codes. -* It can introduce false positives on the testing. For instance, using `prank` to simulate calls from account that is not EOA (e.g. a contract) can allow transactions that are not possible in the blockchain. +* It can introduce false positives on the testing. For instance, using `prank` to simulate calls from a contract can allow transactions that are not possible in the blockchain. -* Using too much cheat codes: +* Using too many cheat codes: * can be confusing or error-prone. Certain cheat code like `prank` allow to change caller in the next external call: It can be difficult to follow, in particular if it is used in internal functions or modifiers. * will create a dependency of your code with the particular tool or cheat code implementation: It can cause produce migrations to other tools or reusing the test code to be more difficult than expected. From 6d770fa2457636472b20ebcbec5ed51f9b782337 Mon Sep 17 00:00:00 2001 From: Gustavo Grieco <31542053+ggrieco-tob@users.noreply.github.com> Date: Fri, 24 Mar 2023 16:38:55 +0100 Subject: [PATCH 5/8] Update on-using-cheat-codes.md --- program-analysis/echidna/advanced/on-using-cheat-codes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/program-analysis/echidna/advanced/on-using-cheat-codes.md b/program-analysis/echidna/advanced/on-using-cheat-codes.md index c4f68d93..9984f519 100644 --- a/program-analysis/echidna/advanced/on-using-cheat-codes.md +++ b/program-analysis/echidna/advanced/on-using-cheat-codes.md @@ -5,7 +5,7 @@ - [How and when to use cheat codes](#how-and-when-to-use-cheat-codes) - [Introduction](#introduction) - [Cheat codes available in Echidna](#cheat-codes-available-in-echidna) - - [Advice on using cheat codes](#advice-on-using-cheat-codes) + - [Advice on the use of cheat codes](#advice-on-the-use-of-cheat-codes) ## Introduction @@ -37,7 +37,7 @@ contract TestPrank { A specific example on the use of `sign` cheat code is available [here in our documentation](hevm-cheats-to-test-permit.md). -## Advice on how and when using cheat codes +## Advice on the use of cheat codes While we provide support for the use of cheat codes, these should be used responsibly. We offer the following advice on the use of cheat codes: From 24cbabcd917863cdd841059910c4afff3c0c2b98 Mon Sep 17 00:00:00 2001 From: ggrieco-tob Date: Fri, 7 Apr 2023 14:21:59 +0200 Subject: [PATCH 6/8] run format --- .../echidna/advanced/on-using-cheat-codes.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/program-analysis/echidna/advanced/on-using-cheat-codes.md b/program-analysis/echidna/advanced/on-using-cheat-codes.md index 9984f519..1f4dcf5d 100644 --- a/program-analysis/echidna/advanced/on-using-cheat-codes.md +++ b/program-analysis/echidna/advanced/on-using-cheat-codes.md @@ -13,7 +13,8 @@ When testing smart contracts in Solidity itself, it can be helpful to use cheat Cheat codes are special functions that allow to change the state of the EVM in ways that are not posible in production. These were introduced by Dapptools in hevm and adopted (and expanded) in other projects such as Foundry. ## Cheat codes available in Echidna -Echidna supports all cheat codes that are available in [hevm](https://github.com/ethereum/hevm). These are documented here: https://hevm.dev/controlling-the-unit-testing-environment.html#cheat-codes. + +Echidna supports all cheat codes that are available in [hevm](https://github.com/ethereum/hevm). These are documented here: https://hevm.dev/controlling-the-unit-testing-environment.html#cheat-codes. If a new cheat code is added in the future, Echidna only needs to update the hevm version and everything should work out of the box. As an example, the `prank` cheat code is able to set the `msg.sender` address in the context of the next external call: @@ -27,7 +28,7 @@ contract TestPrank { address constant HEVM_ADDRESS = 0x7109709ECfa91a80626fF3989D68f67F5b1DD12D; IHevm hevm = IHevm(HEVM_ADDRESS); Contract c = ... - + function prankContract() public payable { hevm.prank(address(0x42424242); c.f(); // `c` will be called with `msg.sender = 0x42424242` @@ -41,10 +42,10 @@ A specific example on the use of `sign` cheat code is available [here in our doc While we provide support for the use of cheat codes, these should be used responsibly. We offer the following advice on the use of cheat codes: -* It can break certain assumptions in Solidity. For example, the compiler assumes that `block.number` is constant during a transaction. There are [reports of the optimizer interfering with (re)computation of the `block.number` or `block.timestamp`](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which can generate incorrect tests when using cheat codes. +- It can break certain assumptions in Solidity. For example, the compiler assumes that `block.number` is constant during a transaction. There are [reports of the optimizer interfering with (re)computation of the `block.number` or `block.timestamp`](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which can generate incorrect tests when using cheat codes. -* It can introduce false positives on the testing. For instance, using `prank` to simulate calls from a contract can allow transactions that are not possible in the blockchain. +- It can introduce false positives on the testing. For instance, using `prank` to simulate calls from a contract can allow transactions that are not possible in the blockchain. -* Using too many cheat codes: - * can be confusing or error-prone. Certain cheat code like `prank` allow to change caller in the next external call: It can be difficult to follow, in particular if it is used in internal functions or modifiers. - * will create a dependency of your code with the particular tool or cheat code implementation: It can cause produce migrations to other tools or reusing the test code to be more difficult than expected. +- Using too many cheat codes: + - can be confusing or error-prone. Certain cheat code like `prank` allow to change caller in the next external call: It can be difficult to follow, in particular if it is used in internal functions or modifiers. + - will create a dependency of your code with the particular tool or cheat code implementation: It can cause produce migrations to other tools or reusing the test code to be more difficult than expected. From 3509316026ea1944ddc25b1243c52281ebc514b3 Mon Sep 17 00:00:00 2001 From: Feist Josselin Date: Sun, 9 Apr 2023 19:24:23 +0200 Subject: [PATCH 7/8] Update SUMMARY.md --- SUMMARY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/SUMMARY.md b/SUMMARY.md index 3e0d5784..83370ca9 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -77,6 +77,7 @@ - [How to detect high gas consumption](./program-analysis/echidna/advanced/finding-transactions-with-high-gas-consumption.md) - [How to perform smart contract fuzzing at a large scale](./program-analysis/echidna/advanced/smart-contract-fuzzing-at-scale.md) - [How to test bytecode-only contracts](./program-analysis/echidna/advanced/testing-bytecode.md) + - [How and when to use cheat codes](program-analysis/echidna/advanced/on-using-cheat-codes.md) - [How to use hevm cheats to test permit](./program-analysis/echidna/advanced/hevm-cheats-to-test-permit.md) - [How to seed Echidna with unit tests](./program-analysis/echidna/advanced/end-to-end-testing.md) - [Understanding and using `multi-abi`](./program-analysis/echidna/advanced/using-multi-abi.md) From d9b8cb0c3cfc2c7c6935c6356636f876be5aef36 Mon Sep 17 00:00:00 2001 From: Feist Josselin Date: Wed, 12 Jul 2023 12:23:37 +0200 Subject: [PATCH 8/8] minor --- program-analysis/echidna/advanced/on-using-cheat-codes.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/program-analysis/echidna/advanced/on-using-cheat-codes.md b/program-analysis/echidna/advanced/on-using-cheat-codes.md index 1f4dcf5d..dcd89771 100644 --- a/program-analysis/echidna/advanced/on-using-cheat-codes.md +++ b/program-analysis/echidna/advanced/on-using-cheat-codes.md @@ -38,13 +38,13 @@ contract TestPrank { A specific example on the use of `sign` cheat code is available [here in our documentation](hevm-cheats-to-test-permit.md). -## Advice on the use of cheat codes +## Risks of cheat codes -While we provide support for the use of cheat codes, these should be used responsibly. We offer the following advice on the use of cheat codes: +While we provide support for the use of cheat codes, these should be used responsibly. Consider that: -- It can break certain assumptions in Solidity. For example, the compiler assumes that `block.number` is constant during a transaction. There are [reports of the optimizer interfering with (re)computation of the `block.number` or `block.timestamp`](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which can generate incorrect tests when using cheat codes. +- Cheat codes can break certain assumptions in Solidity. For example, the compiler assumes that `block.number` is constant during a transaction. There are [reports of the optimizer interfering with (re)computation of the `block.number` or `block.timestamp`](https://github.com/ethereum/solidity/issues/12963#issuecomment-1110162425), which can generate incorrect tests when using cheat codes. -- It can introduce false positives on the testing. For instance, using `prank` to simulate calls from a contract can allow transactions that are not possible in the blockchain. +- Cheat codes can introduce false positives on the testing. For instance, using `prank` to simulate calls from a contract can allow transactions that are not possible in the blockchain. - Using too many cheat codes: - can be confusing or error-prone. Certain cheat code like `prank` allow to change caller in the next external call: It can be difficult to follow, in particular if it is used in internal functions or modifiers.