From ff7366b72a0c2b2d58aee0cf0a7084579a89d6b3 Mon Sep 17 00:00:00 2001 From: Guillermo Larregay <115007237+glarregay-tob@users.noreply.github.com> Date: Tue, 11 Apr 2023 09:40:31 -0300 Subject: [PATCH 1/5] add article about ffi with example --- .../interacting-with-offchain-data-via-ffi.md | 111 ++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md diff --git a/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md new file mode 100644 index 00000000..91faa925 --- /dev/null +++ b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md @@ -0,0 +1,111 @@ +# Interacting with off-chain data using the `ffi` cheatcode + + +## Introduction + +Since the implementation of the HEVM cheat codes in Echidna, it is possible to interact with off-chain data by means of the `ffi` cheatcode. This function allows the caller to execute an arbitrary command on the system running Echidna and read its output, enabling the possibility of getting external data into a fuzzing campaign. + + +## A word of caution + +In general, the usage of cheatcodes is not encouraged, since manipulating the EVM execution environment can lead to unpredictable results and false positives or negatives in fuzzing tests. + +This piece of advice becomes more critical when using `ffi`. This cheatcode basically allows arbitrary code execution on the host system, so it's not just the EVM execution environment that can be manipulated. Running malicious or untrusted tests with `ffi` can have disastrous consequences. + +The usage of this cheatcode should be extremely limited, well documented, and only reserved for cases where there is not a secure alternative. + + +## Pre-requisites + +If reading the previous section didn't scare you enough and you still want to use `ffi`, you will need to explicitly tell Echidna to allow the cheatcode in the tests. This safety measure makes sure you don't accidentally execute `ffi` code. + +To enable the cheatcode, set the 'allowFFI` flag to `true` in your Echidna configuration file: + +```yaml +allowFFI: true +``` + + +## Uses + +Some of the use cases for `ffi` are: + +* Making prices or other information available on-chain during a fuzzing campaign. For example, you can use `ffi` to feed an oracle with "live" data. +* Get randomness in a test. As you know, there is no randomness source on-chain, so using this cheatcode you can get a random value from the device running the fuzz tests. +* Integrate with algorithms not ported to Solidity language, or perform comparisons between two implementations. Some examples for this item include signing and hashing, or custom calculations algorithms. + + +## Example: Call an off-chain program and read its output + +This example will show how to create a simple call to an external executable, passing some values as parameters, and read its output. Keep in mind that the return values of the called program should be an abi-encoded data chunk that can be later decoded via `abi.decode()`. No newlines are allowed in the return values. + +Before digging into the example, there's something else to keep in mind: When interacting with external processes, you will need to convert from Solidity data types to string, to pass values as arguments to the off-chain executable. You can use the [crytic/properties](https://github.com/crytic/properties) helpers for converting. + +For the example we will be creating a python example script that returns a random `uint256` value and a `bytes32` hash calculated from an integer input value. This doesn't represent a "useful" use case, but will be enough to show how the `ffi` cheatcode is used. Finally, we won't perform sanity checks for data types or values, we will just assume the input data will be correct. + +(This script was tested with Python 3.11, Web3 6.0.0 and eth-abi 4.0.0. Some functions had different names in prior versions of the libraries) + +```python +import sys +import secrets +from web3 import Web3 +from eth_abi import encode + +# Usage: python3 script.py number +number = int(sys.argv[1]) + +# Generate a 10-byte random number +random = int(secrets.token_hex(10), 16) + +# Generate the keccak hash of the input value +hashed = Web3.solidity_keccak(['uint256'], [number]) + +# ABI-encode the output +abi_encoded = encode(['uint256', 'bytes32'], [random, hashed]).hex() + +# Make sure that it doesn't print a newline character +print("0x" + abi_encoded, end="") +``` + +You can test this program with various inputs and see what the output is. If it works correctly, the program should output a 512-bit hex string that is the ABI-encoded representation of a 256-bit integer followed by a bytes32. + +Now let's create the Solidity contract that will be run by Echidna to interact with the previous script. + +```solidity +pragma solidity ^0.8.0; + +// HEVM helper +import "@crytic/properties/contracts/util/Hevm.sol"; + +// Helpers to convert uint256 to string +import "@crytic/properties/contracts/util/PropertiesHelper.sol"; + +contract TestFFI { + function test_ffi(uint256 number) public { + + // Prepare the array of executable and parameters + string[] memory inp = new string[](3); + inp[0] = "python3"; + inp[1] = "script.py"; + inp[2] = PropertiesLibString.toString(number); + + // Call the program outside the EVM environment + bytes memory res = hevm.ffi(inp); + + // Decode the return values + (uint256 random, bytes32 hashed) = abi.decode(res, (uint256, bytes32)); + + // Make sure the return value is the expected + bytes32 hashed_solidity = keccak256(abi.encodePacked(number)); + assert(hashed_solidity == hashed); + } +} +``` + +The minimal configuration file for this test is the following: + +```yaml +testMode: "assertion" +allowFFI: true +``` + From 04464910b371ac81e293a5c493fb30991bfb2299 Mon Sep 17 00:00:00 2001 From: Guillermo Larregay <115007237+glarregay-tob@users.noreply.github.com> Date: Tue, 11 Apr 2023 09:40:52 -0300 Subject: [PATCH 2/5] add link to new article in README --- program-analysis/echidna/advanced/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/program-analysis/echidna/advanced/README.md b/program-analysis/echidna/advanced/README.md index 4642f151..d858adcf 100644 --- a/program-analysis/echidna/advanced/README.md +++ b/program-analysis/echidna/advanced/README.md @@ -9,3 +9,4 @@ - [How to use hevm cheats to test permit](./hevm-cheats-to-test-permit.md): How to test code that depends on ecrecover signatures using hevm cheat codes - [How to seed Echidna with unit tests](./end-to-end-testing.md): How to use existing unit tests to seed Echidna - [Understanding and using `multi-abi`](./using-multi-abi.md): What is `multi-abi` testing, and how can it be used +- [Interacting with off-chain data via FFI cheatcode](./interacting-with-offchain-data-via-ffi.md): Using the `ffi` cheatcode as a way of communicating with the operating system. From 0f25882ea54d1286933aeb93f2ea8d9eae70e4b7 Mon Sep 17 00:00:00 2001 From: Guillermo Larregay <115007237+glarregay-tob@users.noreply.github.com> Date: Tue, 11 Apr 2023 10:22:24 -0300 Subject: [PATCH 3/5] linter --- .../interacting-with-offchain-data-via-ffi.md | 19 ++++++------------- 1 file changed, 6 insertions(+), 13 deletions(-) diff --git a/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md index 91faa925..3255c4d8 100644 --- a/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md +++ b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md @@ -1,39 +1,34 @@ # Interacting with off-chain data using the `ffi` cheatcode - ## Introduction Since the implementation of the HEVM cheat codes in Echidna, it is possible to interact with off-chain data by means of the `ffi` cheatcode. This function allows the caller to execute an arbitrary command on the system running Echidna and read its output, enabling the possibility of getting external data into a fuzzing campaign. - ## A word of caution -In general, the usage of cheatcodes is not encouraged, since manipulating the EVM execution environment can lead to unpredictable results and false positives or negatives in fuzzing tests. +In general, the usage of cheatcodes is not encouraged, since manipulating the EVM execution environment can lead to unpredictable results and false positives or negatives in fuzzing tests. This piece of advice becomes more critical when using `ffi`. This cheatcode basically allows arbitrary code execution on the host system, so it's not just the EVM execution environment that can be manipulated. Running malicious or untrusted tests with `ffi` can have disastrous consequences. The usage of this cheatcode should be extremely limited, well documented, and only reserved for cases where there is not a secure alternative. - ## Pre-requisites If reading the previous section didn't scare you enough and you still want to use `ffi`, you will need to explicitly tell Echidna to allow the cheatcode in the tests. This safety measure makes sure you don't accidentally execute `ffi` code. -To enable the cheatcode, set the 'allowFFI` flag to `true` in your Echidna configuration file: +To enable the cheatcode, set the 'allowFFI`flag to`true` in your Echidna configuration file: ```yaml allowFFI: true ``` - ## Uses Some of the use cases for `ffi` are: -* Making prices or other information available on-chain during a fuzzing campaign. For example, you can use `ffi` to feed an oracle with "live" data. -* Get randomness in a test. As you know, there is no randomness source on-chain, so using this cheatcode you can get a random value from the device running the fuzz tests. -* Integrate with algorithms not ported to Solidity language, or perform comparisons between two implementations. Some examples for this item include signing and hashing, or custom calculations algorithms. - +- Making prices or other information available on-chain during a fuzzing campaign. For example, you can use `ffi` to feed an oracle with "live" data. +- Get randomness in a test. As you know, there is no randomness source on-chain, so using this cheatcode you can get a random value from the device running the fuzz tests. +- Integrate with algorithms not ported to Solidity language, or perform comparisons between two implementations. Some examples for this item include signing and hashing, or custom calculations algorithms. ## Example: Call an off-chain program and read its output @@ -69,7 +64,7 @@ print("0x" + abi_encoded, end="") You can test this program with various inputs and see what the output is. If it works correctly, the program should output a 512-bit hex string that is the ABI-encoded representation of a 256-bit integer followed by a bytes32. -Now let's create the Solidity contract that will be run by Echidna to interact with the previous script. +Now let's create the Solidity contract that will be run by Echidna to interact with the previous script. ```solidity pragma solidity ^0.8.0; @@ -82,7 +77,6 @@ import "@crytic/properties/contracts/util/PropertiesHelper.sol"; contract TestFFI { function test_ffi(uint256 number) public { - // Prepare the array of executable and parameters string[] memory inp = new string[](3); inp[0] = "python3"; @@ -108,4 +102,3 @@ The minimal configuration file for this test is the following: testMode: "assertion" allowFFI: true ``` - From e34a9963b7fc2ef03838e6021577f3bd592aaea4 Mon Sep 17 00:00:00 2001 From: Guillermo Larregay <115007237+glarregay-tob@users.noreply.github.com> Date: Tue, 18 Apr 2023 10:29:17 -0300 Subject: [PATCH 4/5] fixes from review --- SUMMARY.md | 1 + .../advanced/interacting-with-offchain-data-via-ffi.md | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/SUMMARY.md b/SUMMARY.md index 3e0d5784..8f5494a0 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -80,6 +80,7 @@ - [How to use hevm cheats to test permit](./program-analysis/echidna/advanced/hevm-cheats-to-test-permit.md) - [How to seed Echidna with unit tests](./program-analysis/echidna/advanced/end-to-end-testing.md) - [Understanding and using `multi-abi`](./program-analysis/echidna/advanced/using-multi-abi.md) + - [Interacting with off-chain data via FFI cheatcode](./program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md) - [Fuzzing tips](./program-analysis/echidna/fuzzing_tips.md) - [Frequently Asked Questions](./program-analysis/echidna/frequently_asked_questions.md) - [Exercises](./program-analysis/echidna/exercises/README.md) diff --git a/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md index 3255c4d8..df5c9cab 100644 --- a/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md +++ b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md @@ -2,7 +2,7 @@ ## Introduction -Since the implementation of the HEVM cheat codes in Echidna, it is possible to interact with off-chain data by means of the `ffi` cheatcode. This function allows the caller to execute an arbitrary command on the system running Echidna and read its output, enabling the possibility of getting external data into a fuzzing campaign. +It is possible for Echidna to interact with off-chain data by means of the `ffi` cheatcode. This function allows the caller to execute an arbitrary command on the system running Echidna and read its output, enabling the possibility of getting external data into a fuzzing campaign. ## A word of caution @@ -34,11 +34,11 @@ Some of the use cases for `ffi` are: This example will show how to create a simple call to an external executable, passing some values as parameters, and read its output. Keep in mind that the return values of the called program should be an abi-encoded data chunk that can be later decoded via `abi.decode()`. No newlines are allowed in the return values. -Before digging into the example, there's something else to keep in mind: When interacting with external processes, you will need to convert from Solidity data types to string, to pass values as arguments to the off-chain executable. You can use the [crytic/properties](https://github.com/crytic/properties) helpers for converting. +Before digging into the example, there's something else to keep in mind: When interacting with external processes, you will need to convert from Solidity data types to string, to pass values as arguments to the off-chain executable. You can use the [crytic/properties](https://github.com/crytic/properties) `toString` [helpers](https://github.com/crytic/properties/blob/main/contracts/util/PropertiesHelper.sol#L447) for converting. For the example we will be creating a python example script that returns a random `uint256` value and a `bytes32` hash calculated from an integer input value. This doesn't represent a "useful" use case, but will be enough to show how the `ffi` cheatcode is used. Finally, we won't perform sanity checks for data types or values, we will just assume the input data will be correct. -(This script was tested with Python 3.11, Web3 6.0.0 and eth-abi 4.0.0. Some functions had different names in prior versions of the libraries) +This script was tested with Python 3.11, Web3 6.0.0 and eth-abi 4.0.0. Some functions had different names in prior versions of the libraries. ```python import sys From 67ad6538451f4b0af69325f4d2843cbe3127ea1b Mon Sep 17 00:00:00 2001 From: Feist Josselin Date: Tue, 11 Jul 2023 13:40:49 +0200 Subject: [PATCH 5/5] Update interacting-with-offchain-data-via-ffi.md --- .../echidna/advanced/interacting-with-offchain-data-via-ffi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md index df5c9cab..8d48571a 100644 --- a/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md +++ b/program-analysis/echidna/advanced/interacting-with-offchain-data-via-ffi.md @@ -16,7 +16,7 @@ The usage of this cheatcode should be extremely limited, well documented, and on If reading the previous section didn't scare you enough and you still want to use `ffi`, you will need to explicitly tell Echidna to allow the cheatcode in the tests. This safety measure makes sure you don't accidentally execute `ffi` code. -To enable the cheatcode, set the 'allowFFI`flag to`true` in your Echidna configuration file: +To enable the cheatcode, set the `allowFFI` flag to `true` in your Echidna configuration file: ```yaml allowFFI: true