From d7fe56578ef915e545d280d4de3fed4a98df314a Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Tue, 1 Mar 2022 19:42:37 +0900 Subject: [PATCH 01/30] fix broken listing --- docs/algorithms/afl/algorithm_ja.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/afl/algorithm_ja.md b/docs/algorithms/afl/algorithm_ja.md index bfd582d8..d6dcfd78 100644 --- a/docs/algorithms/afl/algorithm_ja.md +++ b/docs/algorithms/afl/algorithm_ja.md @@ -31,7 +31,7 @@ fuzzuf afl --in_dir=path/to/initial/seeds/ -- path/to/PUT @@ - `--log_file=path/to/log/file` - ログ出力や、デバッグモードでビルドした場合のデバッグ出力を記録するファイルを指定します。 - 指定されない場合は、標準出力に出力されます。 - - ローカルなオプション(AFLのみで有効) +- ローカルなオプション(AFLのみで有効) - `--dict_file=path/to/dict/file` - 追加の辞書ファイルへのパスを指定します。 From 6dac187dcb3d367f3ab14dd1d0e3914a09c6bb09 Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Wed, 2 Mar 2022 12:42:41 +0900 Subject: [PATCH 02/30] format TODO.md and add To-Dos of IJON --- TODO.md | 39 +++++++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 8 deletions(-) diff --git a/TODO.md b/TODO.md index 1071ad25..a48cb99e 100644 --- a/TODO.md +++ b/TODO.md @@ -130,15 +130,25 @@ With this mode, the performance of fuzzuf's libFuzzer would be comparable to tha It's too bad `Mutator` has some raw pointers as its members, such as `u8 *Mutator::outbuf` and `u8 *Mutator::tmpbuf`. These members can be smart pointers or `std::vector`. We just want to replace them. -### Implement resume mode and parallel fuzzing in AFL +### Add CODING\_RULE.md and refactor the code in accordance with CODING\_RULE.md + +In the past, we didn't have no explicit coding rules. Nevertheless, we have continued developping fuzzuf simultaneously and almost independently of each other. As a result, the code base doesn't look well-organized. This would make the contributors and users confusing, so we must fix it. We have already almost finished creating CODING\_RULE.md internally. We will release it after review and formatting is complete. After Especially, because we started implementing libFuzzer at a very early stage, the large part of the implementation of libFuzzer doesn't conform to that rules. We will resolve this issue gradually simply because they are too large to fix immediately. + + +## To-Dos in each fuzzing algorithm (most of which don't require careful consideration) + +### AFL +This section documents To-Dos of AFL. + +#### Implement resume mode and parallel fuzzing in AFL They are just unimplemented. -### Implement SIGUSR1 Handling on AFL +#### Implement SIGUSR1 Handling on AFL This feature is just unimplemented. -### Remove careless templates from AFL +#### Remove careless templates from AFL In the implmentation of AFL, we use a lot of `template` to allow users to define the derived classes of `AFLTestcase` and `AFLState`. But this is just cutting corners. Let us explain what we've done with an example. Let's say, we want to define a function that takes a reference of some struct as an argument. The struct has a member named "x". The function would look like the following: @@ -168,21 +178,34 @@ void SomeFunc(const SomeStruct& stru) { We should rewrite the classes of AFL in the same way eventually. +### IJON + +This section documents To-Dos of IJON. + +#### Implement annotations + +What IJON proposed is not just a fuzzer, but a set of a fuzzer and an annotation mechanism in PUTs. +Unfortunately, the annotation mechanism is not implemented in fuzzuf because it doesn't have its own instrumentation tool yet. +This should be implemented immediately after fuzzuf-cc becomes ready. + +#### Test with Super Mario Bros. + +To prove that our IJON fuzzer works well to some extent, one of the most comprehensible tests would be check if the fuzzer can play Super Mario Bros. well, as done in the paper of IJON. + ### Nautilus -This section documents To-Dos of the Nautilus mode. + +This section documents To-Dos of Nautilus. #### Use vector instead of string + The current parser/unparser of the grammar and rules uses `std::string` as its data pool instead of `std::vector`. This should be changed to `std::vector` because `std::string` is originally not meant to hold unprintable strings. #### Improve queue + The implementation of the seed queue in the original Nautilus has a lot of room for optimization. The current implementation of fuzzuf is similar to the original one and should be improved. -### Add CODING\_RULE.md and refactor the code in accordance with CODING\_RULE.md - -In the past, we didn't have no explicit coding rules. Nevertheless, we have continued developping fuzzuf simultaneously and almost independently of each other. As a result, the code base doesn't look well-organized. This would make the contributors and users confusing, so we must fix it. We have already almost finished creating CODING\_RULE.md internally. We will release it after review and formatting is complete. After Especially, because we started implementing libFuzzer at a very early stage, the large part of the implementation of libFuzzer doesn't conform to that rules. We will resolve this issue gradually simply because they are too large to fix immediately. - [^mopt]: Chenyang Lyu, Shouling Ji, Chao Zhang, Yuwei Li, Wei-Han Lee, Yu Song, and Raheem Beyah. 2019. MOpt: Optimized Mutation Scheduling for Fuzzers. In Proceedings of the 28th USENIX Security Symposium (Security'19). [^eclipser]: Jaeseung Choi, Joonun Jang, Choongwoo Han, and Sang K. Cha. 2019. Grey-box Concolic Testing on Binary Code. In Proceedings of the 41st ACM/IEEE International Conference on Software Engineering (ICSE'19). [^qsym]: Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018. QSYM : A Practical Concolic Execution Engine Tailored for Hybrid Fuzzing. In Proceedings of the 27th USENIX Security Symposium (Security'18). From ed3fe07dd17496f678324e82db2c1c881446982d Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Wed, 2 Mar 2022 12:42:54 +0900 Subject: [PATCH 03/30] add Japanese documents of IJON --- docs/algorithms/ijon/algorithm_ja.md | 50 +++++++++++++++++++++++ docs/algorithms/ijon/implementation_ja.md | 24 +++++++++++ 2 files changed, 74 insertions(+) create mode 100644 docs/algorithms/ijon/algorithm_ja.md create mode 100644 docs/algorithms/ijon/implementation_ja.md diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md new file mode 100644 index 00000000..515874c5 --- /dev/null +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -0,0 +1,50 @@ +# IJON + +## IJONとは + +[IJON](https://github.com/RUB-SysSec/ijon/)[^ijon]は、[SysSec](https://informatik.rub.de/syssec/)によって提案された、PUTがファザーに新しいフィードバックを返せるようにするアノテーション手法、およびそれらのフィードバックに対応したファザーです。有名なファザーの多くは、PUTからコードカバレッジをフィードバックとして受け取ることでプログラムの新しい挙動を見つけようとする、カバレッジガイデッドファザーに種別されます。典型的なカバレッジガイデットファザーには以下のような弱点があります: + +- コードカバレッジがどのような順序によって獲得されたかについて気にしていない。例えば、「関数Aが実行された直後に関数Bが実行されること」が引き起こす条件となっているようなバグがあったとする。ファザーは、バグを引き起こす入力と「関数Bの実行の後に関数Aの実行を引き起こすような入力」を区別できないため、後者のみを試してしまい、バグを見落とす可能性がある。より突き詰めていえば、そもそも「関数Aと関数Bの両方を通らせる入力」と「関数Aを通らせる入力および関数Bを通らせる入力のセット」を区別できず、どちらか一方を既に試している場合には、もう一方を試さないようなアルゴリズムが多い。 + - コードカバレッジの種類の中で、パスカバレッジを用いた場合には、この問題にはある程度対処できる。しかし、実行パスが異なる入力を過剰に保存した場合には、同じような入力を保持する可能性が高くなり、ファジングキャンペーン全体の効率が低下してしまうというトレードオフがあり、そのトレードオフを自動で良い塩梅に調節するというのは難しい。 +- コードカバレッジでは気づくことのできない内部状態の変化が存在しえる。例えば、IJONの論文に記載されている通り、ゲームにおけるプレイヤーの座標などを考えてみると分かりやすい。プレイヤーが画面上のどの位置に存在しているかは、ゲームの新しい状態を発見する上で重要になる可能性が高い。プレイヤーが画面右下にいるよりも画面左上にいるほうが、新しいイベントの発生座標に近いかもしれない。しかしながら、コードカバレッジだけをフィードバックとしている場合は、どちらの座標にいても同じフィードバックが返されることになる。 + +IJONは、これらの問題点に対処できるシンプルな解決策として、「PUTに対して人の手でアノテーションを行う」という方法を提案しています。PUTをソースコードからビルドしカバレッジの取得を計装する際、人間がソースコードにアノテーションを加えることで、PUTがファザーに与えるフィードバックをカスタマイズできます。IJONが提供するアノテーションには様々なものがあり、人間が重要な内部状態だと考えるものを明示するために用います。例えば「ある変数の最大値をフィードバックに記録する」、「ある2変数の差の最小値を記録する」といったアノテーションが可能です。 + +実際には、IJONはAFLをベースとして実装されているため、PUTが返すフィードバックは、(Hashed) Edge Coverageであり、共有メモリを経由してファザーに受け渡されます。したがって、ソースコードに記述できるアノテーションは、具体的には共有メモリに対して値を書き込む関数およびマクロとして実装されています。これらのマクロや関数は、計装ツールがEdge Coverageを計装する際に、一緒にコンパイルされます。 + +このようにIJONは、AFLベースかつ実用的なファジングにおいて求められるハーネス記述用のインターフェイスを備えており、fuzzufの応用可能性の向上を目的としてfuzzuf上に実装されています。 + +## CLI上での使用方法 + +fuzzufをインストールした状態で、 + +```bash +fuzzuf ijon -i path/to/initial/seeds/ path/to/PUT @@ +``` + +で起動できます。指定可能なグローバルなオプションはAFLと同様です。 +AFLのオプションについては[AFL/algorithm_ja.md](/docs/algorithms/afl/algorithm_ja.md)を参照してください。 + +使用できるIJONのローカルオプションは以下です: + +- `--forksrv 0|1` + - 1が指定された場合、fork server modeが有効になります。 デフォルトで有効です。 + + +## アルゴリズム概要 + +IJONはAFLの処理をほとんど残したまま、処理を追加する形で実装されています。大まかには、AFLとの差分は以下のとおりです: + +- havocミューテーションのいくつかのケースが改変されている。 +- AFLのシードキューとは別に、IJON専用のシードキューを持つ。 + - IJONのシードキューには、「共有メモリ上の64bit非負整数の配列のそれぞれの要素について、最も大きい値を記録させたシード」が保存される。 +- ファジングループの冒頭で、ランダムに処理の分岐が発生する。 + - 80%の確率でIJONのシードキューからシードが選択される。この場合、直後にhavocステージに移行し、havocミューテーションを一定回数行うとファジングループの先頭に戻る。 + - 20%の確率でAFLのシードキューからシードが選択される。この場合は、元のAFLと同じ流れでミューテーションを行う。 +- PUT実行後の、PUTから得られたフィードバックを元にしたシードキューの更新では、IJONのシードキューも更新する。 + - 20%の確率でAFLが選択された場合でも、IJONのシードキューは更新する。 +- 定数の値がいくつか変更されている。 + +## 参考文献 + +[^ijon]: C. Aschermann, S. Schumilo, A. Abbasi, and T. Holz. 2020. IJON: Exploring Deep State Spaces via Fuzzing. In Proceedings of the 41st IEEE Symposium on Security and Privacy (S&P’20). diff --git a/docs/algorithms/ijon/implementation_ja.md b/docs/algorithms/ijon/implementation_ja.md new file mode 100644 index 00000000..c936539e --- /dev/null +++ b/docs/algorithms/ijon/implementation_ja.md @@ -0,0 +1,24 @@ +# fuzzufにおけるIJONの実装 + +## 参考にしたオリジナルのIJONの実装 + +- バージョン: 2.51b-ijon +- コミット: https://github.com/RUB-SysSec/ijon/commit/56ebfe34709dd93f5da7871624ce6eadacc3ae4c + +## オリジナルの実装との差分 + +バグなどによって意図しない差異が発生していない限り、オリジナルのファザーを完全に再現しており、差分はない。 +ただし、fuzzuf上のAFLが実装していないオリジナルのAFLの機能については、当然IJONにおいても使用できない。 + +## 重要なTo-Do: アノテーションの実装 + +注意しなければならないこととして、現在のfuzzufではIJONの主要なコンポーネントであるはずのアノテーションが実装されていません。 +これは、fuzzufがまだ独自の計装ツールを持っていないことに起因しており、計装ツールが準備され次第、実装に取り掛かる予定です。 +その他のTo-Doについては、[TODO.md](https://github.com/fuzzuf/fuzzuf/blob/master/TODO.md)を確認してください。 + +## 追加されているHierarFlowルーチン + +- SelectSeed: IJONのシードキューからシードを選択する。 +- PrintAflIsSelected: 20%の確率でAFLのコードフローが選択された場合に、AFLが選択された旨を出力する。 +- MaxHavoc: 80%の確率でIJONのコードフローが選択された場合に開始するミューテーション。 +- UpdateMax: IJONのシードキューを更新する処理。 From 90350d60204d96c05e0351142b52d488e8a6bdba Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Wed, 2 Mar 2022 12:54:41 +0900 Subject: [PATCH 04/30] format TODO.md more --- TODO.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/TODO.md b/TODO.md index a48cb99e..c486ecb4 100644 --- a/TODO.md +++ b/TODO.md @@ -132,7 +132,7 @@ It's too bad `Mutator` has some raw pointers as its members, such as `u8 *Mutato ### Add CODING\_RULE.md and refactor the code in accordance with CODING\_RULE.md -In the past, we didn't have no explicit coding rules. Nevertheless, we have continued developping fuzzuf simultaneously and almost independently of each other. As a result, the code base doesn't look well-organized. This would make the contributors and users confusing, so we must fix it. We have already almost finished creating CODING\_RULE.md internally. We will release it after review and formatting is complete. After Especially, because we started implementing libFuzzer at a very early stage, the large part of the implementation of libFuzzer doesn't conform to that rules. We will resolve this issue gradually simply because they are too large to fix immediately. +In the past, we didn't have no explicit coding rules. Nevertheless, we have continued developping fuzzuf simultaneously and almost independently of each other. As a result, the code base doesn't look well-organized. This would make the contributors and users confusing, so we must fix it. We have already almost finished creating CODING\_RULE.md internally. We will release it after review and formatting is complete. Especially, because we started implementing libFuzzer at a very early stage, the large part of the implementation of libFuzzer doesn't conform to that rules. We will resolve this issue gradually simply because they are too large to fix immediately. ## To-Dos in each fuzzing algorithm (most of which don't require careful consideration) @@ -152,7 +152,7 @@ This feature is just unimplemented. In the implmentation of AFL, we use a lot of `template` to allow users to define the derived classes of `AFLTestcase` and `AFLState`. But this is just cutting corners. Let us explain what we've done with an example. Let's say, we want to define a function that takes a reference of some struct as an argument. The struct has a member named "x". The function would look like the following: -``` +```cpp void SomeFunc(const SomeStruct& stru) { std::cout << stru.x << std::endl; } @@ -160,7 +160,7 @@ void SomeFunc(const SomeStruct& stru) { Next, we would like to generalize this function so that it can accept similar struct types. Specificallt, we should be able to pass to the function the instances of other structs that have the member "x". Obviously, we can do that in the following way: -``` +```cpp template void SomeFunc(const Struct& stru) { std::cout << stru.x << std::endl; @@ -169,7 +169,7 @@ void SomeFunc(const Struct& stru) { But, another possible solution would be to define the virtual member function `SomeStruct::GetX()`, and to make other structs derive it. Like this way: -``` +```cpp // Define SomeStruct::GetX() in advance void SomeFunc(const SomeStruct& stru) { std::cout << stru.GetX() << std::endl; @@ -185,7 +185,7 @@ This section documents To-Dos of IJON. #### Implement annotations What IJON proposed is not just a fuzzer, but a set of a fuzzer and an annotation mechanism in PUTs. -Unfortunately, the annotation mechanism is not implemented in fuzzuf because it doesn't have its own instrumentation tool yet. +Unfortunately, the annotation mechanism is not implemented in fuzzuf because fuzzuf doesn't have its own instrumentation tool yet. This should be implemented immediately after fuzzuf-cc becomes ready. #### Test with Super Mario Bros. From bd12b321fab6f06314c86a1314a884618f55a9bd Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Wed, 2 Mar 2022 13:09:48 +0900 Subject: [PATCH 05/30] lint IJON documents --- docs/algorithms/ijon/algorithm_ja.md | 4 ++-- docs/algorithms/ijon/implementation_ja.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index 515874c5..bf2be69f 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -2,10 +2,10 @@ ## IJONとは -[IJON](https://github.com/RUB-SysSec/ijon/)[^ijon]は、[SysSec](https://informatik.rub.de/syssec/)によって提案された、PUTがファザーに新しいフィードバックを返せるようにするアノテーション手法、およびそれらのフィードバックに対応したファザーです。有名なファザーの多くは、PUTからコードカバレッジをフィードバックとして受け取ることでプログラムの新しい挙動を見つけようとする、カバレッジガイデッドファザーに種別されます。典型的なカバレッジガイデットファザーには以下のような弱点があります: +[IJON](https://github.com/RUB-SysSec/ijon/)[^ijon]は、[SysSec](https://informatik.rub.de/syssec/)によって提案された、PUTがファザーに新しいフィードバックを返せるようにするアノテーション手法、およびそれらのフィードバックに対応したファザーです。有名なファザーの多くは、PUTからコードカバレッジをフィードバックとして受け取ることでプログラムの新しい挙動を見つけようとする、カバレッジガイデッドファザーに種別されます。典型的なカバレッジガイデッドファザーには以下のような弱点があります: - コードカバレッジがどのような順序によって獲得されたかについて気にしていない。例えば、「関数Aが実行された直後に関数Bが実行されること」が引き起こす条件となっているようなバグがあったとする。ファザーは、バグを引き起こす入力と「関数Bの実行の後に関数Aの実行を引き起こすような入力」を区別できないため、後者のみを試してしまい、バグを見落とす可能性がある。より突き詰めていえば、そもそも「関数Aと関数Bの両方を通らせる入力」と「関数Aを通らせる入力および関数Bを通らせる入力のセット」を区別できず、どちらか一方を既に試している場合には、もう一方を試さないようなアルゴリズムが多い。 - - コードカバレッジの種類の中で、パスカバレッジを用いた場合には、この問題にはある程度対処できる。しかし、実行パスが異なる入力を過剰に保存した場合には、同じような入力を保持する可能性が高くなり、ファジングキャンペーン全体の効率が低下してしまうというトレードオフがあり、そのトレードオフを自動で良い塩梅に調節するというのは難しい。 + - コードカバレッジの種類の中で、パスカバレッジを用いた場合には、この問題にはある程度対処できる。しかし、実行パスが異なる入力を過剰に保存してしまった場合には、同じファジング結果になる似たような入力を保持しやすくなり、ファジングキャンペーン全体の効率が低下してしまうというトレードオフがある。このトレードオフを自動で良い塩梅に調節するというのは難しい。 - コードカバレッジでは気づくことのできない内部状態の変化が存在しえる。例えば、IJONの論文に記載されている通り、ゲームにおけるプレイヤーの座標などを考えてみると分かりやすい。プレイヤーが画面上のどの位置に存在しているかは、ゲームの新しい状態を発見する上で重要になる可能性が高い。プレイヤーが画面右下にいるよりも画面左上にいるほうが、新しいイベントの発生座標に近いかもしれない。しかしながら、コードカバレッジだけをフィードバックとしている場合は、どちらの座標にいても同じフィードバックが返されることになる。 IJONは、これらの問題点に対処できるシンプルな解決策として、「PUTに対して人の手でアノテーションを行う」という方法を提案しています。PUTをソースコードからビルドしカバレッジの取得を計装する際、人間がソースコードにアノテーションを加えることで、PUTがファザーに与えるフィードバックをカスタマイズできます。IJONが提供するアノテーションには様々なものがあり、人間が重要な内部状態だと考えるものを明示するために用います。例えば「ある変数の最大値をフィードバックに記録する」、「ある2変数の差の最小値を記録する」といったアノテーションが可能です。 diff --git a/docs/algorithms/ijon/implementation_ja.md b/docs/algorithms/ijon/implementation_ja.md index c936539e..75954633 100644 --- a/docs/algorithms/ijon/implementation_ja.md +++ b/docs/algorithms/ijon/implementation_ja.md @@ -7,8 +7,8 @@ ## オリジナルの実装との差分 -バグなどによって意図しない差異が発生していない限り、オリジナルのファザーを完全に再現しており、差分はない。 -ただし、fuzzuf上のAFLが実装していないオリジナルのAFLの機能については、当然IJONにおいても使用できない。 +バグなどによって意図しない差異が発生していない限り、オリジナルのファザーを完全に再現しており、差分はありません。 +ただし、fuzzuf上のAFLが実装していない、オリジナルのAFLの機能については、当然IJONにおいても使用できません。 ## 重要なTo-Do: アノテーションの実装 From 0991d3efc8a9c476678a0dc6daa756834d59a640 Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Wed, 2 Mar 2022 15:14:16 +0900 Subject: [PATCH 06/30] add English IJON documents and modify Japanese ones accordingly --- docs/algorithms/ijon/algorithm_en.md | 45 +++++++++++++++++++++++ docs/algorithms/ijon/algorithm_ja.md | 2 +- docs/algorithms/ijon/implementation_en.md | 24 ++++++++++++ docs/algorithms/ijon/implementation_ja.md | 2 +- 4 files changed, 71 insertions(+), 2 deletions(-) create mode 100644 docs/algorithms/ijon/algorithm_en.md create mode 100644 docs/algorithms/ijon/implementation_en.md diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md new file mode 100644 index 00000000..1574dd15 --- /dev/null +++ b/docs/algorithms/ijon/algorithm_en.md @@ -0,0 +1,45 @@ +# IJON + +## What is IJON? + +IJON is an annotation mechanism that allows PUTs to return new types of feedback to fuzzers, and a fuzzer that supports those feedbacks, proposed by SysSec. Many famous fuzzers are classified as coverage-guided fuzzers, which try to find new behaviors in a program by receiving code coverage as feedback from PUTs. A typical coverage-guided fuzzer has the following weaknesses: + +- They do not care about the order in which the code coverage was obtained. For example, suppose there is a bug whose triggering condition is "function B must be executed immediately after function A is executed". Because the fuzzer cannot distinguish between the input that causes the bug and the input that causes the execution of function A after the execution of function B, it may try only the latter and overlook the bug. More to the point, many algorithms cannot distinguish between "input that causes both function A and function B to be called" and "a set of input that causes function A to be called and input that causes function B to be called" in the first place, and if one of the two is already tested, the other won't be tested. + - Among the types of code coverage, path coverage can deal with this problem to some extent. However, there is a trade-off in that if a fuzzer over-stores inputs with different execution paths, it is more likely to retain similar inputs that result in the same fuzzing result, and the overall efficiency of a fuzzing campaign will be reduced. It is difficult to automatically adjust this trade-off to a good degree. +- There can be internal state changes that cannot be noticed by code coverage. For example, as described in IJON's paper, consider the coordinates of the player in a game. The position of the player on the screen is likely to be important in discovering new states of the game. If the player is in the upper-left corner of the screen, he may be closer to the coordinates of the new event than if in the lower right corner of the screen. However, if only code coverage is used as feedback, the same feedback will be returned regardless of which coordinate the player is at. + +IJON proposes a simple solution to these problems: human annotation on PUTs. When building a PUT from source code and instrumenting it to obtain coverage, humans can add annotations to the source code to customize the feedback that the PUT gives to a fuzzer. There are various annotations provided by IJON that humans can use to specify what they consider to be important internal states. For example, annotations such as "record the maximum value of a variable in the feedback" or "record the minimum difference between two variables" are possible. + +In practice, because IJON is implemented based on AFL, the feedback returned by PUT is (Hashed) Edge Coverage, which is passed on to the fuzzer via shared memory. Therefore, the annotations that can be written in the source code are specifically implemented as functions and macros that write values to the shared memory. These macros and functions are compiled together when the instrumentation tools instrument the Edge Coverage. +Thus, because IJON is an AFL-based fuzzer and has an interface for harness description required in practical fuzzing, it has been implemented on fuzzuf to improve the applicability of fuzzuf. + + +## How to use fuzzuf's IJON CLI + +With `fuzzuf` installed, run + +```bash +fuzzuf ijon -i path/to/initial/seeds/ path/to/PUT @@ +``` + +to start IJON. The global options that can be specified are the same as for AFL. +For AFL options, see [AFL/algorithm_en.md](/docs/algorithms/afl/algorithm_en.md). + +The local options for IJON that can be used are: + +- `--forksrv 0|1` + - If 1 is specified, then fork server mode is enabled. It is enabled by default. + +## Algorithm Overview + +IJON is implemented in a way that retains most of the functions of AFL, and adds additional functions. Roughly speaking, the differences from AFL are as follows: + +- Some cases of havoc mutation have been modified. +- IJON has its own seed queue, apart from the AFL seed queue. + - For each element of a 64-bit non-negative integer array in shared memory, the IJON seed queue saves the seed that makes a program record the largest value in the element. +- At the beginning of the fuzzing loop, the procedure branches randomly. + - 80% of the time, a seed is selected from the IJON seed queue. In this case, it immediately moves to the havoc stage, and returns to the beginning of the fuzzing loop after a certain number of havoc mutations. + - 20% of the time, a seed is selected from the AFL seed queue. In this case, mutation is performed in the same flow as the original AFL. +- After a PUT is executed, the IJON seed queue is updated based on the feedback obtained from the PUT. + - Even when AFL is selected in 20% probability, the IJON seed queue is also updated. +- Some of the constant values have been changed. diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index bf2be69f..aa6e2a3e 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -37,7 +37,7 @@ IJONはAFLの処理をほとんど残したまま、処理を追加する形で - havocミューテーションのいくつかのケースが改変されている。 - AFLのシードキューとは別に、IJON専用のシードキューを持つ。 - - IJONのシードキューには、「共有メモリ上の64bit非負整数の配列のそれぞれの要素について、最も大きい値を記録させたシード」が保存される。 + - 共有メモリ上の64bit非負整数の配列のそれぞれの要素について、IJONのシードキューは、「その要素へ最も大きい値を、PUTに記録させたシード」を保存する。 - ファジングループの冒頭で、ランダムに処理の分岐が発生する。 - 80%の確率でIJONのシードキューからシードが選択される。この場合、直後にhavocステージに移行し、havocミューテーションを一定回数行うとファジングループの先頭に戻る。 - 20%の確率でAFLのシードキューからシードが選択される。この場合は、元のAFLと同じ流れでミューテーションを行う。 diff --git a/docs/algorithms/ijon/implementation_en.md b/docs/algorithms/ijon/implementation_en.md new file mode 100644 index 00000000..a5b8bb88 --- /dev/null +++ b/docs/algorithms/ijon/implementation_en.md @@ -0,0 +1,24 @@ +# IJON implementation in fuzzuf + +## Reference IJON Implementation + +- Version: 2.51b-ijon +- Commit: https://github.com/RUB-SysSec/ijon/commit/56ebfe34709dd93f5da7871624ce6eadacc3ae4c + +## Differences from the original implementation + +Unless there are unintended discrepancies due to bugs, the original implementation has been completely reproduced and there are no differences between it and ours. +However, the functions of the original AFL, which are not implemented in fuzzuf AFL, of course cannot be used in IJON. + +## Important To-Do: Implementing Annotations + +It should be noted that annotations, which must be a major component of IJON, are not implemented in fuzzuf at present. +This is due to the fact that fuzzuf does not have its own instrumentation tool yet, and therefore we will start implementing it as soon as the instrumentation tool is ready. +For other To-Dos, please check [TODO.md](https://github.com/fuzzuf/fuzzuf/blob/master/TODO.md). + +## HierarFlow routines that have been added + +- SelectSeed: selects a seed from the IJON seed queue. +- PrintAflIsSelected: prints that AFL is selected when AFL's code flow is selected with 20% probability. +- MaxHavoc: starts the havoc stage when IJON's code flow is selected with 80% probability. +- UpdateMax: updates the IJON seed queue. diff --git a/docs/algorithms/ijon/implementation_ja.md b/docs/algorithms/ijon/implementation_ja.md index 75954633..875628ef 100644 --- a/docs/algorithms/ijon/implementation_ja.md +++ b/docs/algorithms/ijon/implementation_ja.md @@ -20,5 +20,5 @@ - SelectSeed: IJONのシードキューからシードを選択する。 - PrintAflIsSelected: 20%の確率でAFLのコードフローが選択された場合に、AFLが選択された旨を出力する。 -- MaxHavoc: 80%の確率でIJONのコードフローが選択された場合に開始するミューテーション。 +- MaxHavoc: 80%の確率でIJONのコードフローが選択された場合に開始されるhavocミューテーション。 - UpdateMax: IJONのシードキューを更新する処理。 From de87b710001f4b13cce57eb4757be0b189081e27 Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Wed, 2 Mar 2022 15:20:06 +0900 Subject: [PATCH 07/30] minor fixes --- docs/algorithms/ijon/algorithm_en.md | 2 +- docs/algorithms/ijon/algorithm_ja.md | 6 +++--- docs/algorithms/ijon/implementation_en.md | 2 +- docs/algorithms/ijon/implementation_ja.md | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 1574dd15..83bed328 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -38,7 +38,7 @@ IJON is implemented in a way that retains most of the functions of AFL, and adds - IJON has its own seed queue, apart from the AFL seed queue. - For each element of a 64-bit non-negative integer array in shared memory, the IJON seed queue saves the seed that makes a program record the largest value in the element. - At the beginning of the fuzzing loop, the procedure branches randomly. - - 80% of the time, a seed is selected from the IJON seed queue. In this case, it immediately moves to the havoc stage, and returns to the beginning of the fuzzing loop after a certain number of havoc mutations. + - 80% of the time, a seed is selected from the IJON seed queue. In this case, the fuzzer immediately moves to the havoc stage, and returns to the beginning of the fuzzing loop after a certain number of havoc mutations. - 20% of the time, a seed is selected from the AFL seed queue. In this case, mutation is performed in the same flow as the original AFL. - After a PUT is executed, the IJON seed queue is updated based on the feedback obtained from the PUT. - Even when AFL is selected in 20% probability, the IJON seed queue is also updated. diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index aa6e2a3e..97026895 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -16,14 +16,14 @@ IJONは、これらの問題点に対処できるシンプルな解決策とし ## CLI上での使用方法 -fuzzufをインストールした状態で、 +`fuzzuf`をインストールした状態で、 ```bash fuzzuf ijon -i path/to/initial/seeds/ path/to/PUT @@ ``` で起動できます。指定可能なグローバルなオプションはAFLと同様です。 -AFLのオプションについては[AFL/algorithm_ja.md](/docs/algorithms/afl/algorithm_ja.md)を参照してください。 +AFLのオプションについては[AFL/algorithm_ja.md#cli上での使用方法](/docs/algorithms/afl/algorithm_ja.md#cli上での使用方法)を参照してください。 使用できるIJONのローカルオプションは以下です: @@ -41,7 +41,7 @@ IJONはAFLの処理をほとんど残したまま、処理を追加する形で - ファジングループの冒頭で、ランダムに処理の分岐が発生する。 - 80%の確率でIJONのシードキューからシードが選択される。この場合、直後にhavocステージに移行し、havocミューテーションを一定回数行うとファジングループの先頭に戻る。 - 20%の確率でAFLのシードキューからシードが選択される。この場合は、元のAFLと同じ流れでミューテーションを行う。 -- PUT実行後の、PUTから得られたフィードバックを元にしたシードキューの更新では、IJONのシードキューも更新する。 +- PUT実行後の、PUTから得られたフィードバックを元にしたシードキューの更新で、IJONのシードキューも更新する。 - 20%の確率でAFLが選択された場合でも、IJONのシードキューは更新する。 - 定数の値がいくつか変更されている。 diff --git a/docs/algorithms/ijon/implementation_en.md b/docs/algorithms/ijon/implementation_en.md index a5b8bb88..5c39bd51 100644 --- a/docs/algorithms/ijon/implementation_en.md +++ b/docs/algorithms/ijon/implementation_en.md @@ -8,7 +8,7 @@ ## Differences from the original implementation Unless there are unintended discrepancies due to bugs, the original implementation has been completely reproduced and there are no differences between it and ours. -However, the functions of the original AFL, which are not implemented in fuzzuf AFL, of course cannot be used in IJON. +However, the functions of the original AFL, which are not implemented in fuzzuf AFL, cannot be used of course also in IJON. ## Important To-Do: Implementing Annotations diff --git a/docs/algorithms/ijon/implementation_ja.md b/docs/algorithms/ijon/implementation_ja.md index 875628ef..217328c3 100644 --- a/docs/algorithms/ijon/implementation_ja.md +++ b/docs/algorithms/ijon/implementation_ja.md @@ -7,7 +7,7 @@ ## オリジナルの実装との差分 -バグなどによって意図しない差異が発生していない限り、オリジナルのファザーを完全に再現しており、差分はありません。 +バグなどによって意図しない差異が発生していない限り、オリジナルの実装を完全に再現しており、差分はありません。 ただし、fuzzuf上のAFLが実装していない、オリジナルのAFLの機能については、当然IJONにおいても使用できません。 ## 重要なTo-Do: アノテーションの実装 From 3c8f53205a688b288e67b5efa84a8a5b3e071c16 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:17:06 +0900 Subject: [PATCH 08/30] Reflect review1 Co-authored-by: Yudai --- docs/algorithms/ijon/implementation_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/implementation_en.md b/docs/algorithms/ijon/implementation_en.md index 5c39bd51..cc9d0aca 100644 --- a/docs/algorithms/ijon/implementation_en.md +++ b/docs/algorithms/ijon/implementation_en.md @@ -12,7 +12,7 @@ However, the functions of the original AFL, which are not implemented in fuzzuf ## Important To-Do: Implementing Annotations -It should be noted that annotations, which must be a major component of IJON, are not implemented in fuzzuf at present. +Be noted that annotation, which is one of the major components of IJON, is not implemented in fuzzuf at present. This is due to the fact that fuzzuf does not have its own instrumentation tool yet, and therefore we will start implementing it as soon as the instrumentation tool is ready. For other To-Dos, please check [TODO.md](https://github.com/fuzzuf/fuzzuf/blob/master/TODO.md). From c691dd771e3b98052e74e10286adfde615c4a385 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:21:53 +0900 Subject: [PATCH 09/30] Reflect review2 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_ja.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index 97026895..c92f3298 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -6,7 +6,7 @@ - コードカバレッジがどのような順序によって獲得されたかについて気にしていない。例えば、「関数Aが実行された直後に関数Bが実行されること」が引き起こす条件となっているようなバグがあったとする。ファザーは、バグを引き起こす入力と「関数Bの実行の後に関数Aの実行を引き起こすような入力」を区別できないため、後者のみを試してしまい、バグを見落とす可能性がある。より突き詰めていえば、そもそも「関数Aと関数Bの両方を通らせる入力」と「関数Aを通らせる入力および関数Bを通らせる入力のセット」を区別できず、どちらか一方を既に試している場合には、もう一方を試さないようなアルゴリズムが多い。 - コードカバレッジの種類の中で、パスカバレッジを用いた場合には、この問題にはある程度対処できる。しかし、実行パスが異なる入力を過剰に保存してしまった場合には、同じファジング結果になる似たような入力を保持しやすくなり、ファジングキャンペーン全体の効率が低下してしまうというトレードオフがある。このトレードオフを自動で良い塩梅に調節するというのは難しい。 -- コードカバレッジでは気づくことのできない内部状態の変化が存在しえる。例えば、IJONの論文に記載されている通り、ゲームにおけるプレイヤーの座標などを考えてみると分かりやすい。プレイヤーが画面上のどの位置に存在しているかは、ゲームの新しい状態を発見する上で重要になる可能性が高い。プレイヤーが画面右下にいるよりも画面左上にいるほうが、新しいイベントの発生座標に近いかもしれない。しかしながら、コードカバレッジだけをフィードバックとしている場合は、どちらの座標にいても同じフィードバックが返されることになる。 +- コードカバレッジでは気づくことのできない内部状態の変化が存在しうる。例えば、IJONの論文に記載されている通り、ゲームにおけるプレイヤーの座標などを考えてみると分かりやすい。プレイヤーが画面上のどの位置に存在しているかは、ゲームの新しい状態を発見する上で重要になる可能性が高い。プレイヤーが画面右下にいるよりも画面左上にいるほうが、新しいイベントの発生座標に近いかもしれない。しかしながら、コードカバレッジだけをフィードバックとしている場合は、どちらの座標にいても同じフィードバックが返されることになる。 IJONは、これらの問題点に対処できるシンプルな解決策として、「PUTに対して人の手でアノテーションを行う」という方法を提案しています。PUTをソースコードからビルドしカバレッジの取得を計装する際、人間がソースコードにアノテーションを加えることで、PUTがファザーに与えるフィードバックをカスタマイズできます。IJONが提供するアノテーションには様々なものがあり、人間が重要な内部状態だと考えるものを明示するために用います。例えば「ある変数の最大値をフィードバックに記録する」、「ある2変数の差の最小値を記録する」といったアノテーションが可能です。 From 9836bd69d875291da937eebf41ac68c6595f1f4a Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:37:16 +0900 Subject: [PATCH 10/30] Reflect review3 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 83bed328..33d59159 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -22,7 +22,7 @@ With `fuzzuf` installed, run fuzzuf ijon -i path/to/initial/seeds/ path/to/PUT @@ ``` -to start IJON. The global options that can be specified are the same as for AFL. +to start IJON. The global options available are the same as for AFL. For AFL options, see [AFL/algorithm_en.md](/docs/algorithms/afl/algorithm_en.md). The local options for IJON that can be used are: From 6b1878ad2343662ccc0e177237db5ace22c2ac34 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:37:44 +0900 Subject: [PATCH 11/30] Reflect review4 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 33d59159..f661468c 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -25,7 +25,7 @@ fuzzuf ijon -i path/to/initial/seeds/ path/to/PUT @@ to start IJON. The global options available are the same as for AFL. For AFL options, see [AFL/algorithm_en.md](/docs/algorithms/afl/algorithm_en.md). -The local options for IJON that can be used are: +The local option for IJON is: - `--forksrv 0|1` - If 1 is specified, then fork server mode is enabled. It is enabled by default. From a0cdab6f5d8e328f7a3285f06ed6ec88fab4199b Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:37:58 +0900 Subject: [PATCH 12/30] Reflect review5 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index f661468c..b292d90c 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -42,4 +42,4 @@ IJON is implemented in a way that retains most of the functions of AFL, and adds - 20% of the time, a seed is selected from the AFL seed queue. In this case, mutation is performed in the same flow as the original AFL. - After a PUT is executed, the IJON seed queue is updated based on the feedback obtained from the PUT. - Even when AFL is selected in 20% probability, the IJON seed queue is also updated. -- Some of the constant values have been changed. +- Some of the constants are changed. From f00f93bb9f5b12658077075bb67a24fd114ab8ea Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:38:27 +0900 Subject: [PATCH 13/30] Reflect review6 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index b292d90c..21b42984 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -40,6 +40,6 @@ IJON is implemented in a way that retains most of the functions of AFL, and adds - At the beginning of the fuzzing loop, the procedure branches randomly. - 80% of the time, a seed is selected from the IJON seed queue. In this case, the fuzzer immediately moves to the havoc stage, and returns to the beginning of the fuzzing loop after a certain number of havoc mutations. - 20% of the time, a seed is selected from the AFL seed queue. In this case, mutation is performed in the same flow as the original AFL. -- After a PUT is executed, the IJON seed queue is updated based on the feedback obtained from the PUT. +- After a PUT exits, the IJON seed queue is updated based on the feedback obtained from the PUT. - Even when AFL is selected in 20% probability, the IJON seed queue is also updated. - Some of the constants are changed. From 559a2125aa770a28619e2a81c5229a8edb8831e8 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:40:28 +0900 Subject: [PATCH 14/30] Reflect review7 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 21b42984..dbf7afc9 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -34,7 +34,7 @@ The local option for IJON is: IJON is implemented in a way that retains most of the functions of AFL, and adds additional functions. Roughly speaking, the differences from AFL are as follows: -- Some cases of havoc mutation have been modified. +- Some cases of havoc mutation are modified. - IJON has its own seed queue, apart from the AFL seed queue. - For each element of a 64-bit non-negative integer array in shared memory, the IJON seed queue saves the seed that makes a program record the largest value in the element. - At the beginning of the fuzzing loop, the procedure branches randomly. From 8a15b432a420dba2262799b8da597465c54481f3 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:41:36 +0900 Subject: [PATCH 15/30] Reflect review7 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_ja.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index c92f3298..a7489538 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -4,7 +4,7 @@ [IJON](https://github.com/RUB-SysSec/ijon/)[^ijon]は、[SysSec](https://informatik.rub.de/syssec/)によって提案された、PUTがファザーに新しいフィードバックを返せるようにするアノテーション手法、およびそれらのフィードバックに対応したファザーです。有名なファザーの多くは、PUTからコードカバレッジをフィードバックとして受け取ることでプログラムの新しい挙動を見つけようとする、カバレッジガイデッドファザーに種別されます。典型的なカバレッジガイデッドファザーには以下のような弱点があります: -- コードカバレッジがどのような順序によって獲得されたかについて気にしていない。例えば、「関数Aが実行された直後に関数Bが実行されること」が引き起こす条件となっているようなバグがあったとする。ファザーは、バグを引き起こす入力と「関数Bの実行の後に関数Aの実行を引き起こすような入力」を区別できないため、後者のみを試してしまい、バグを見落とす可能性がある。より突き詰めていえば、そもそも「関数Aと関数Bの両方を通らせる入力」と「関数Aを通らせる入力および関数Bを通らせる入力のセット」を区別できず、どちらか一方を既に試している場合には、もう一方を試さないようなアルゴリズムが多い。 +- コードカバレッジがどのような順序によって獲得されたかについて気にしていない。例えば、「関数Aが実行された直後に関数Bが実行されること」が引き起こす条件となっているようなバグがあったとする。ファザーは、バグを引き起こす入力と「関数Bの実行の後に関数Aの実行を引き起こすような入力」を区別できないため、後者のみを試してしまい、バグを見落とす可能性がある。より突き詰めていえば、そもそも「関数Aと関数Bの両方を通らせる入力」と「関数Aを通らせる入力および関数Bを通らせる入力の２つ」を区別できず、どちらか一方を既に試している場合には、もう一方を試さないようなアルゴリズムが多い。 - コードカバレッジの種類の中で、パスカバレッジを用いた場合には、この問題にはある程度対処できる。しかし、実行パスが異なる入力を過剰に保存してしまった場合には、同じファジング結果になる似たような入力を保持しやすくなり、ファジングキャンペーン全体の効率が低下してしまうというトレードオフがある。このトレードオフを自動で良い塩梅に調節するというのは難しい。 - コードカバレッジでは気づくことのできない内部状態の変化が存在しうる。例えば、IJONの論文に記載されている通り、ゲームにおけるプレイヤーの座標などを考えてみると分かりやすい。プレイヤーが画面上のどの位置に存在しているかは、ゲームの新しい状態を発見する上で重要になる可能性が高い。プレイヤーが画面右下にいるよりも画面左上にいるほうが、新しいイベントの発生座標に近いかもしれない。しかしながら、コードカバレッジだけをフィードバックとしている場合は、どちらの座標にいても同じフィードバックが返されることになる。 From 68f9663e904207f72a1f42200f666a1f77fe4594 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Wed, 2 Mar 2022 17:55:42 +0900 Subject: [PATCH 16/30] Reflect review8 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_ja.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index a7489538..c8c6ad99 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -37,7 +37,7 @@ IJONはAFLの処理をほとんど残したまま、処理を追加する形で - havocミューテーションのいくつかのケースが改変されている。 - AFLのシードキューとは別に、IJON専用のシードキューを持つ。 - - 共有メモリ上の64bit非負整数の配列のそれぞれの要素について、IJONのシードキューは、「その要素へ最も大きい値を、PUTに記録させたシード」を保存する。 + - 共有メモリ上の64bit非負整数の配列のそれぞれの要素について、IJONのシードキューは「その要素へ最も大きい値を記録したシード」を保存する。 - ファジングループの冒頭で、ランダムに処理の分岐が発生する。 - 80%の確率でIJONのシードキューからシードが選択される。この場合、直後にhavocステージに移行し、havocミューテーションを一定回数行うとファジングループの先頭に戻る。 - 20%の確率でAFLのシードキューからシードが選択される。この場合は、元のAFLと同じ流れでミューテーションを行う。 From 53ffa8d7e2e308eaf02cf3fcba9e3f0a6c5504e7 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Thu, 3 Mar 2022 11:12:13 +0900 Subject: [PATCH 17/30] Reflect review9 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index dbf7afc9..81106112 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -2,7 +2,7 @@ ## What is IJON? -IJON is an annotation mechanism that allows PUTs to return new types of feedback to fuzzers, and a fuzzer that supports those feedbacks, proposed by SysSec. Many famous fuzzers are classified as coverage-guided fuzzers, which try to find new behaviors in a program by receiving code coverage as feedback from PUTs. A typical coverage-guided fuzzer has the following weaknesses: +[IJON](https://github.com/RUB-SysSec/ijon/)[^ijon] is an annotation mechanism that allows PUTs to return new types of feedback to fuzzers, and a fuzzer that supports those feedback, proposed by [SysSec](https://informatik.rub.de/syssec/). Many famous fuzzers are classified as coverage-guided fuzzers, which try to find new behavior in a program by receiving code coverage as feedback from PUTs. A typical coverage-guided fuzzer has the following weaknesses: - They do not care about the order in which the code coverage was obtained. For example, suppose there is a bug whose triggering condition is "function B must be executed immediately after function A is executed". Because the fuzzer cannot distinguish between the input that causes the bug and the input that causes the execution of function A after the execution of function B, it may try only the latter and overlook the bug. More to the point, many algorithms cannot distinguish between "input that causes both function A and function B to be called" and "a set of input that causes function A to be called and input that causes function B to be called" in the first place, and if one of the two is already tested, the other won't be tested. - Among the types of code coverage, path coverage can deal with this problem to some extent. However, there is a trade-off in that if a fuzzer over-stores inputs with different execution paths, it is more likely to retain similar inputs that result in the same fuzzing result, and the overall efficiency of a fuzzing campaign will be reduced. It is difficult to automatically adjust this trade-off to a good degree. From 92230fa0faa9ad5c9250d40dbe40002c92dbff24 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Thu, 3 Mar 2022 11:15:43 +0900 Subject: [PATCH 18/30] Reflect review10 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 81106112..f701ecc1 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -5,7 +5,7 @@ [IJON](https://github.com/RUB-SysSec/ijon/)[^ijon] is an annotation mechanism that allows PUTs to return new types of feedback to fuzzers, and a fuzzer that supports those feedback, proposed by [SysSec](https://informatik.rub.de/syssec/). Many famous fuzzers are classified as coverage-guided fuzzers, which try to find new behavior in a program by receiving code coverage as feedback from PUTs. A typical coverage-guided fuzzer has the following weaknesses: - They do not care about the order in which the code coverage was obtained. For example, suppose there is a bug whose triggering condition is "function B must be executed immediately after function A is executed". Because the fuzzer cannot distinguish between the input that causes the bug and the input that causes the execution of function A after the execution of function B, it may try only the latter and overlook the bug. More to the point, many algorithms cannot distinguish between "input that causes both function A and function B to be called" and "a set of input that causes function A to be called and input that causes function B to be called" in the first place, and if one of the two is already tested, the other won't be tested. - - Among the types of code coverage, path coverage can deal with this problem to some extent. However, there is a trade-off in that if a fuzzer over-stores inputs with different execution paths, it is more likely to retain similar inputs that result in the same fuzzing result, and the overall efficiency of a fuzzing campaign will be reduced. It is difficult to automatically adjust this trade-off to a good degree. + - Among the types of code coverage, path coverage can deal with this problem to some extent. However, there is a trade-off in that if a fuzzer over-stores inputs with different execution paths, it is more likely to retain similar inputs that yield the same fuzzing result, which will eventually reduce the overall efficiency of a fuzzing campaign. It is difficult to adjust this trade-off automatically. - There can be internal state changes that cannot be noticed by code coverage. For example, as described in IJON's paper, consider the coordinates of the player in a game. The position of the player on the screen is likely to be important in discovering new states of the game. If the player is in the upper-left corner of the screen, he may be closer to the coordinates of the new event than if in the lower right corner of the screen. However, if only code coverage is used as feedback, the same feedback will be returned regardless of which coordinate the player is at. IJON proposes a simple solution to these problems: human annotation on PUTs. When building a PUT from source code and instrumenting it to obtain coverage, humans can add annotations to the source code to customize the feedback that the PUT gives to a fuzzer. There are various annotations provided by IJON that humans can use to specify what they consider to be important internal states. For example, annotations such as "record the maximum value of a variable in the feedback" or "record the minimum difference between two variables" are possible. From ab66ec9e101560d8caf4b15de3d1e29063a7ca09 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Thu, 3 Mar 2022 11:16:25 +0900 Subject: [PATCH 19/30] Reflect review11 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index f701ecc1..a04228c2 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -6,7 +6,7 @@ - They do not care about the order in which the code coverage was obtained. For example, suppose there is a bug whose triggering condition is "function B must be executed immediately after function A is executed". Because the fuzzer cannot distinguish between the input that causes the bug and the input that causes the execution of function A after the execution of function B, it may try only the latter and overlook the bug. More to the point, many algorithms cannot distinguish between "input that causes both function A and function B to be called" and "a set of input that causes function A to be called and input that causes function B to be called" in the first place, and if one of the two is already tested, the other won't be tested. - Among the types of code coverage, path coverage can deal with this problem to some extent. However, there is a trade-off in that if a fuzzer over-stores inputs with different execution paths, it is more likely to retain similar inputs that yield the same fuzzing result, which will eventually reduce the overall efficiency of a fuzzing campaign. It is difficult to adjust this trade-off automatically. -- There can be internal state changes that cannot be noticed by code coverage. For example, as described in IJON's paper, consider the coordinates of the player in a game. The position of the player on the screen is likely to be important in discovering new states of the game. If the player is in the upper-left corner of the screen, he may be closer to the coordinates of the new event than if in the lower right corner of the screen. However, if only code coverage is used as feedback, the same feedback will be returned regardless of which coordinate the player is at. +- There may be some internal state changes that the code coverage cannot reveal. For example, as described in IJON's paper, consider the coordinates of the player in a game. The position of the player on the screen is likely to be important in discovering new states of the game. If the player is in the upper-left corner of the screen, he may be closer to the coordinates of the new event than if in the lower right corner of the screen. However, if the fuzzer just uses the code coverage, both of them will produce the same feedback regardless of which coordinate the player is at. IJON proposes a simple solution to these problems: human annotation on PUTs. When building a PUT from source code and instrumenting it to obtain coverage, humans can add annotations to the source code to customize the feedback that the PUT gives to a fuzzer. There are various annotations provided by IJON that humans can use to specify what they consider to be important internal states. For example, annotations such as "record the maximum value of a variable in the feedback" or "record the minimum difference between two variables" are possible. From b4af63e85cd6fcfc1bfb87d612aab6edd1ba4225 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Thu, 3 Mar 2022 11:33:20 +0900 Subject: [PATCH 20/30] Reflect review12 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index a04228c2..2a2d2e88 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -36,7 +36,7 @@ IJON is implemented in a way that retains most of the functions of AFL, and adds - Some cases of havoc mutation are modified. - IJON has its own seed queue, apart from the AFL seed queue. - - For each element of a 64-bit non-negative integer array in shared memory, the IJON seed queue saves the seed that makes a program record the largest value in the element. + - For each element of a 64-bit non-negative integer array in shared memory, the IJON seed queue saves the seed that made a program record the maximum value in the element. - At the beginning of the fuzzing loop, the procedure branches randomly. - 80% of the time, a seed is selected from the IJON seed queue. In this case, the fuzzer immediately moves to the havoc stage, and returns to the beginning of the fuzzing loop after a certain number of havoc mutations. - 20% of the time, a seed is selected from the AFL seed queue. In this case, mutation is performed in the same flow as the original AFL. From 81bd1898cf2830aee9a75fbb1348a59fc3c344aa Mon Sep 17 00:00:00 2001 From: potetisensei Date: Thu, 3 Mar 2022 11:34:16 +0900 Subject: [PATCH 21/30] Reflect review13 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_ja.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index c8c6ad99..4dcfeed8 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -10,7 +10,7 @@ IJONは、これらの問題点に対処できるシンプルな解決策として、「PUTに対して人の手でアノテーションを行う」という方法を提案しています。PUTをソースコードからビルドしカバレッジの取得を計装する際、人間がソースコードにアノテーションを加えることで、PUTがファザーに与えるフィードバックをカスタマイズできます。IJONが提供するアノテーションには様々なものがあり、人間が重要な内部状態だと考えるものを明示するために用います。例えば「ある変数の最大値をフィードバックに記録する」、「ある2変数の差の最小値を記録する」といったアノテーションが可能です。 -実際には、IJONはAFLをベースとして実装されているため、PUTが返すフィードバックは、(Hashed) Edge Coverageであり、共有メモリを経由してファザーに受け渡されます。したがって、ソースコードに記述できるアノテーションは、具体的には共有メモリに対して値を書き込む関数およびマクロとして実装されています。これらのマクロや関数は、計装ツールがEdge Coverageを計装する際に、一緒にコンパイルされます。 +実際には、IJONはAFLをベースとして実装されているため、PUTが返すフィードバックは、(Hashed) Edge Coverageであり、共有メモリを経由してファザーに受け渡されます。したがって、IJONは、ソースコードに記述できるアノテーションを、具体的には共有メモリに対して値を書き込む関数およびマクロとして実装しています。これらのマクロや関数は、計装ツールがEdge Coverageを計装する際に、一緒にコンパイルされます。 このようにIJONは、AFLベースかつ実用的なファジングにおいて求められるハーネス記述用のインターフェイスを備えており、fuzzufの応用可能性の向上を目的としてfuzzuf上に実装されています。 From 9cb1f0fc4b4731b2c5f175a8d0f5d46cce4e090e Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Thu, 3 Mar 2022 17:24:26 +0900 Subject: [PATCH 22/30] add example usage --- docs/algorithms/ijon/algorithm_en.md | 61 +++++++++++++++++++++++++--- docs/algorithms/ijon/algorithm_ja.md | 57 +++++++++++++++++++++++++- 2 files changed, 111 insertions(+), 7 deletions(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 2a2d2e88..913c3aaf 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -10,19 +10,21 @@ IJON proposes a simple solution to these problems: human annotation on PUTs. When building a PUT from source code and instrumenting it to obtain coverage, humans can add annotations to the source code to customize the feedback that the PUT gives to a fuzzer. There are various annotations provided by IJON that humans can use to specify what they consider to be important internal states. For example, annotations such as "record the maximum value of a variable in the feedback" or "record the minimum difference between two variables" are possible. -In practice, because IJON is implemented based on AFL, the feedback returned by PUT is (Hashed) Edge Coverage, which is passed on to the fuzzer via shared memory. Therefore, the annotations that can be written in the source code are specifically implemented as functions and macros that write values to the shared memory. These macros and functions are compiled together when the instrumentation tools instrument the Edge Coverage. -Thus, because IJON is an AFL-based fuzzer and has an interface for harness description required in practical fuzzing, it has been implemented on fuzzuf to improve the applicability of fuzzuf. - +In practice, because IJON's fuzzer is implemented based on AFL, the feedback returned by PUT is (Hashed) Edge Coverage, which is passed on to the fuzzer via shared memory. Therefore, the annotations that can be written in the source code are specifically implemented as functions and macros that write values to the shared memory. These macros and functions are compiled together when the instrumentation tools instrument the Edge Coverage. +Thus, because IJON has an AFL-based fuzzer and an interface for harness description required in practical fuzzing, it has been implemented on fuzzuf to improve the applicability of fuzzuf. ## How to use fuzzuf's IJON CLI -With `fuzzuf` installed, run +To use IJON's fuzzer, first you need to prepare annotated PUTs with instrumentation tools. +Because fuzzuf doesn't have its own instrumentation tool, please visit [IJON's repo](https://github.com/RUB-SysSec/ijon/) and build the original instrumentation tool. + +After you create a PUT and install `fuzzuf`, run ```bash fuzzuf ijon -i path/to/initial/seeds/ path/to/PUT @@ ``` -to start IJON. The global options available are the same as for AFL. +to start IJON's fuzzer. The global options available are the same as for AFL. For AFL options, see [AFL/algorithm_en.md](/docs/algorithms/afl/algorithm_en.md). The local option for IJON is: @@ -30,6 +32,55 @@ The local option for IJON is: - `--forksrv 0|1` - If 1 is specified, then fork server mode is enabled. It is enabled by default. +## Example Usage + +You can test the original instrumentation tool and IJON's fuzzer in fuzzuf by building and fuzzing [test.c](https://github.com/RUB-SysSec/ijon/blob/master/test.c) and [test2.c](https://github.com/RUB-SysSec/ijon/blob/master/test2.c) found in IJON's repo. Note that, test.c, included in the latest commit (56ebfe34), may yield compilation errors and in that case you need to apply the following changes: + +```diff +diff --git a/llvm_mode/afl-rt.h b/llvm_mode/afl-rt.h +index 616cbd8..28d5f9d 100644 +--- a/llvm_mode/afl-rt.h ++++ b/llvm_mode/afl-rt.h +@@ -45,14 +45,14 @@ void ijon_enable_feedback(); + void ijon_disable_feedback(); + + #define _IJON_CONCAT(x, y) x##y +-#define _IJON_UNIQ_NAME() IJON_CONCAT(temp,__LINE__) ++#define _IJON_UNIQ_NAME IJON_CONCAT(temp,__LINE__) + #define _IJON_ABS_DIST(x,y) ((x)<(y) ? (y)-(x) : (x)-(y)) + + #define IJON_BITS(x) ((x==0)?{0}:__builtin_clz(x)) + #define IJON_INC(x) ijon_map_inc(ijon_hashstr(__LINE__,__FILE__)^(x)) + #define IJON_SET(x) ijon_map_set(ijon_hashstr(__LINE__,__FILE__)^(x)) + +-#define IJON_CTX(x) ({ uint32_t hash = hashstr(__LINE__,__FILE__); ijon_xor_state(hash); __typeof__(x) IJON_UNIQ_NAME() = (x); ijon_xor_state(hash); IJON_UNIQ_NAME(); }) ++#define IJON_CTX(x) ({ uint32_t hash = ijon_hashstr(__LINE__,__FILE__); ijon_xor_state(hash); __typeof__(x) IJON_UNIQ_NAME = (x); ijon_xor_state(hash); IJON_UNIQ_NAME; }) + + #define IJON_MAX(x) ijon_max(ijon_hashstr(__LINE__,__FILE__),(x)) + #define IJON_MIN(x) ijon_max(ijon_hashstr(__LINE__,__FILE__),0xffffffffffffffff-(x)) +diff --git a/test.c b/test.c +index 50b1b05..aa022f6 100644 +--- a/test.c ++++ b/test.c +@@ -3,6 +3,7 @@ + #include + #include + #include ++#include +``` + +For example, you can build test.c and fuzz the produced binary with the following commands: + +```bash +$ (path_to_ijon)/llvm_mode/afl-clang-fast (path_to_ijon)/test.c -o test +$ mkdir /tmp/ijon_test_indir/ && echo hello > /tmp/ijon_test_indir/hello +$ fuzzuf ijon -i /tmp/ijon_test_indir/ ./test +``` + +Here, you don't need to specify `@@` in the last command because the binary receives inputs via stdin. + +While test.c and test2.c gives you an idea how you can use annotations, you can check README and source code in IJON's repo to understand their further usage. + ## Algorithm Overview IJON is implemented in a way that retains most of the functions of AFL, and adds additional functions. Roughly speaking, the differences from AFL are as follows: diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index 4dcfeed8..6798532f 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -10,13 +10,16 @@ IJONは、これらの問題点に対処できるシンプルな解決策として、「PUTに対して人の手でアノテーションを行う」という方法を提案しています。PUTをソースコードからビルドしカバレッジの取得を計装する際、人間がソースコードにアノテーションを加えることで、PUTがファザーに与えるフィードバックをカスタマイズできます。IJONが提供するアノテーションには様々なものがあり、人間が重要な内部状態だと考えるものを明示するために用います。例えば「ある変数の最大値をフィードバックに記録する」、「ある2変数の差の最小値を記録する」といったアノテーションが可能です。 -実際には、IJONはAFLをベースとして実装されているため、PUTが返すフィードバックは、(Hashed) Edge Coverageであり、共有メモリを経由してファザーに受け渡されます。したがって、IJONは、ソースコードに記述できるアノテーションを、具体的には共有メモリに対して値を書き込む関数およびマクロとして実装しています。これらのマクロや関数は、計装ツールがEdge Coverageを計装する際に、一緒にコンパイルされます。 +実際には、IJONのファザーはAFLをベースとして実装されているため、PUTが返すフィードバックは、(Hashed) Edge Coverageであり、共有メモリを経由してファザーに受け渡されます。したがって、IJONは、ソースコードに記述できるアノテーションを、具体的には共有メモリに対して値を書き込む関数およびマクロとして実装しています。これらのマクロや関数は、計装ツールがEdge Coverageを計装する際に、一緒にコンパイルされます。 このようにIJONは、AFLベースかつ実用的なファジングにおいて求められるハーネス記述用のインターフェイスを備えており、fuzzufの応用可能性の向上を目的としてfuzzuf上に実装されています。 ## CLI上での使用方法 -`fuzzuf`をインストールした状態で、 +IJONのファザーを利用するには、まず、計装ツールを用いてアノテーションをほどこしたPUTを準備する必要があります。 +fuzzufには計装ツールが存在していないため、[IJONのリポジトリ](https://github.com/RUB-SysSec/ijon/)からオリジナルの計装ツールをビルドしてください。 + +IJONのファザーは、ビルドした計装ツールによってPUTを作成した後、`fuzzuf`をインストールした状態で、 ```bash fuzzuf ijon -i path/to/initial/seeds/ path/to/PUT @@ @@ -31,6 +34,56 @@ AFLのオプションについては[AFL/algorithm_ja.md#cli上での使用方 - 1が指定された場合、fork server modeが有効になります。 デフォルトで有効です。 +## 使用例 + +ビルドした計装ツールおよびfuzzufに実装されたIJONのファザーをテストする簡単な方法は、IJONのリポジトリにある[test.c](https://github.com/RUB-SysSec/ijon/blob/master/test.c)および[test2.c](https://github.com/RUB-SysSec/ijon/blob/master/test2.c)をビルドし、ファジングしてみることです。ただし、test.cは、現在の最新コミット(56ebfe34)では正常にビルドできないため、以下の変更を加えてください。 + +```diff +diff --git a/llvm_mode/afl-rt.h b/llvm_mode/afl-rt.h +index 616cbd8..28d5f9d 100644 +--- a/llvm_mode/afl-rt.h ++++ b/llvm_mode/afl-rt.h +@@ -45,14 +45,14 @@ void ijon_enable_feedback(); + void ijon_disable_feedback(); + + #define _IJON_CONCAT(x, y) x##y +-#define _IJON_UNIQ_NAME() IJON_CONCAT(temp,__LINE__) ++#define _IJON_UNIQ_NAME IJON_CONCAT(temp,__LINE__) + #define _IJON_ABS_DIST(x,y) ((x)<(y) ? (y)-(x) : (x)-(y)) + + #define IJON_BITS(x) ((x==0)?{0}:__builtin_clz(x)) + #define IJON_INC(x) ijon_map_inc(ijon_hashstr(__LINE__,__FILE__)^(x)) + #define IJON_SET(x) ijon_map_set(ijon_hashstr(__LINE__,__FILE__)^(x)) + +-#define IJON_CTX(x) ({ uint32_t hash = hashstr(__LINE__,__FILE__); ijon_xor_state(hash); __typeof__(x) IJON_UNIQ_NAME() = (x); ijon_xor_state(hash); IJON_UNIQ_NAME(); }) ++#define IJON_CTX(x) ({ uint32_t hash = ijon_hashstr(__LINE__,__FILE__); ijon_xor_state(hash); __typeof__(x) IJON_UNIQ_NAME = (x); ijon_xor_state(hash); IJON_UNIQ_NAME; }) + + #define IJON_MAX(x) ijon_max(ijon_hashstr(__LINE__,__FILE__),(x)) + #define IJON_MIN(x) ijon_max(ijon_hashstr(__LINE__,__FILE__),0xffffffffffffffff-(x)) +diff --git a/test.c b/test.c +index 50b1b05..aa022f6 100644 +--- a/test.c ++++ b/test.c +@@ -3,6 +3,7 @@ + #include + #include + #include ++#include +``` + +例えば、test.cをビルドしたバイナリは、以下のようにしてファジングできます: + +```bash +$ (path_to_ijon)/llvm_mode/afl-clang-fast (path_to_ijon)/test.c -o test +$ mkdir /tmp/ijon_test_indir/ && echo hello > /tmp/ijon_test_indir/hello +$ fuzzuf ijon -i /tmp/ijon_test_indir/ ./test +``` + +test.cは標準入力から入力を受け付けるため、`@@`を指定する必要がないことに注意してください。 + +test.cおよびtest2.cを見れば、アノテーションの付け方のイメージを掴むことができますが、 +より詳しいアノテーションの使い方については、IJONのREADMEおよびソースコードを参照してください。 + ## アルゴリズム概要 IJONはAFLの処理をほとんど残したまま、処理を追加する形で実装されています。大まかには、AFLとの差分は以下のとおりです: From 546141de3ca344088cf8569d8f03ea9bd8e444ff Mon Sep 17 00:00:00 2001 From: potetisensei Date: Thu, 3 Mar 2022 17:26:34 +0900 Subject: [PATCH 23/30] Reflect review14 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 913c3aaf..22b7baba 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -4,7 +4,7 @@ [IJON](https://github.com/RUB-SysSec/ijon/)[^ijon] is an annotation mechanism that allows PUTs to return new types of feedback to fuzzers, and a fuzzer that supports those feedback, proposed by [SysSec](https://informatik.rub.de/syssec/). Many famous fuzzers are classified as coverage-guided fuzzers, which try to find new behavior in a program by receiving code coverage as feedback from PUTs. A typical coverage-guided fuzzer has the following weaknesses: -- They do not care about the order in which the code coverage was obtained. For example, suppose there is a bug whose triggering condition is "function B must be executed immediately after function A is executed". Because the fuzzer cannot distinguish between the input that causes the bug and the input that causes the execution of function A after the execution of function B, it may try only the latter and overlook the bug. More to the point, many algorithms cannot distinguish between "input that causes both function A and function B to be called" and "a set of input that causes function A to be called and input that causes function B to be called" in the first place, and if one of the two is already tested, the other won't be tested. +- It does not care about the order in which it obtained the code coverage. For example, suppose there is a bug whose triggering condition is "executing function B immediately after executing function A". Because the fuzzer cannot distinguish between the input that causes the bug and the one that causes the execution of function A after function B, it may try only the latter and overlook the bug. More to the point, many algorithms cannot distinguish between "the input that causes both function A and function B to be called" and "two inputs such that one of them executes only function A and the other executes only function B." If the latter two inputs are tested first, the fuzzer will not test the former one. - Among the types of code coverage, path coverage can deal with this problem to some extent. However, there is a trade-off in that if a fuzzer over-stores inputs with different execution paths, it is more likely to retain similar inputs that yield the same fuzzing result, which will eventually reduce the overall efficiency of a fuzzing campaign. It is difficult to adjust this trade-off automatically. - There may be some internal state changes that the code coverage cannot reveal. For example, as described in IJON's paper, consider the coordinates of the player in a game. The position of the player on the screen is likely to be important in discovering new states of the game. If the player is in the upper-left corner of the screen, he may be closer to the coordinates of the new event than if in the lower right corner of the screen. However, if the fuzzer just uses the code coverage, both of them will produce the same feedback regardless of which coordinate the player is at. From da3558e7d53cc3c7b426308f793f03a71d5ef172 Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Thu, 3 Mar 2022 17:30:22 +0900 Subject: [PATCH 24/30] Reflect review15 --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index 22b7baba..aa467728 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -10,7 +10,7 @@ IJON proposes a simple solution to these problems: human annotation on PUTs. When building a PUT from source code and instrumenting it to obtain coverage, humans can add annotations to the source code to customize the feedback that the PUT gives to a fuzzer. There are various annotations provided by IJON that humans can use to specify what they consider to be important internal states. For example, annotations such as "record the maximum value of a variable in the feedback" or "record the minimum difference between two variables" are possible. -In practice, because IJON's fuzzer is implemented based on AFL, the feedback returned by PUT is (Hashed) Edge Coverage, which is passed on to the fuzzer via shared memory. Therefore, the annotations that can be written in the source code are specifically implemented as functions and macros that write values to the shared memory. These macros and functions are compiled together when the instrumentation tools instrument the Edge Coverage. +In practice, because IJON's fuzzer is implemented based on AFL, the feedback returned by PUT is (Hashed) Edge Coverage, which is passed on to the fuzzer via shared memory. Therefore, IJON specifically implemented the annotations, which users can write in the source code, as functions and macros that write values to the shared memory. These macros and functions are compiled together when the instrumentation tools instrument the Edge Coverage. Thus, because IJON has an AFL-based fuzzer and an interface for harness description required in practical fuzzing, it has been implemented on fuzzuf to improve the applicability of fuzzuf. ## How to use fuzzuf's IJON CLI From f6f46fa3fa50cda5a55d041d48ecb75abe1daaa3 Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Thu, 3 Mar 2022 17:40:05 +0900 Subject: [PATCH 25/30] add IJON in README --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 81824b53..0c7c9df7 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,7 @@ Note, when using fuzzuf from CLI, you have to separate global options (options a |---|---|---|---|---|--- |AFL|Greybox|A re-implementation of general purpose fuzzer, representing a CGF. Also available as a template for its derivatives.|[How to use fuzzuf's AFL CLI](/docs/algorithms/afl/algorithm_en.md#how-to-use-fuzzufs-afl-cli)|[Algorithm Overview](/docs/algorithms/afl/algorithm_en.md#algorithm-overview)|:white_check_mark: |AFLFast|Greybox|An implementation of AFLFast, utilizing an AFL template.
The algorithm tries to increase its performance by manipulating the power schedule.|[CLI Usage](/docs/algorithms/aflfast/algorithm_en.md#cli-usage)|[Algorithm Overview](/docs/algorithms/aflfast/algorithm_en.md#algorithm-overview)|:white_check_mark: +|IJON|Greybox|An implementation of IJON, utilizing an AFL template.
A fuzzer that can fuzz PUTs in an *internal-state-aware* manner with manual annotations to PUTs.|[CLI Usage](/docs/algorithms/ijon/algorithm_en.md#how-to-use-fuzzufs-ijon-cli)|[Algorithm Overview](/docs/algorithms/ijon/algorithm_en.md#algorithm-overview) |VUzzer|Greybox|A mutation-based fuzzer guess data structures by analyzing the PUT control flow and the data flow.|Read [Prerequisite](/docs/algorithms/vuzzer/algorithm_en.md#prerequisite) first, then [Usage on CLI](docs/algorithms/vuzzer/algorithm_en.md#usage-on-cli)|[Algorithm Overview](/docs/algorithms/vuzzer/algorithm_en.md#algorithm-overview) |libFuzzer|Greybox|CGF included in the LLVM project's compiler-rt libraries.|[How to use libFuzzer on fuzzuf](/docs/algorithms/libfuzzer/manual.md#how-to-use-libfuzzer-on-fuzzuf)|[What is libFuzzer?](/docs/algorithms/libfuzzer/algorithm_en.md#what-is-libfuzzer) |Nezha|Greybox|A fuzzer originates from libFuzzer that tries to find defects in the program by executing programs having different implementations with the same input and compares its execution results (differential fuzzing).|[How to use Nezha on fuzzuf](/docs/algorithms/nezha/manual.md#how-to-use-nezha-on-fuzzuf)|TBD From be5834355476e838d1b61a0e8c93b594459a02b8 Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Thu, 3 Mar 2022 17:45:50 +0900 Subject: [PATCH 26/30] remove a redundant line from IJON description in README --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0c7c9df7..b4ea31b3 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,7 @@ Note, when using fuzzuf from CLI, you have to separate global options (options a |---|---|---|---|---|--- |AFL|Greybox|A re-implementation of general purpose fuzzer, representing a CGF. Also available as a template for its derivatives.|[How to use fuzzuf's AFL CLI](/docs/algorithms/afl/algorithm_en.md#how-to-use-fuzzufs-afl-cli)|[Algorithm Overview](/docs/algorithms/afl/algorithm_en.md#algorithm-overview)|:white_check_mark: |AFLFast|Greybox|An implementation of AFLFast, utilizing an AFL template.
The algorithm tries to increase its performance by manipulating the power schedule.|[CLI Usage](/docs/algorithms/aflfast/algorithm_en.md#cli-usage)|[Algorithm Overview](/docs/algorithms/aflfast/algorithm_en.md#algorithm-overview)|:white_check_mark: -|IJON|Greybox|An implementation of IJON, utilizing an AFL template.
A fuzzer that can fuzz PUTs in an *internal-state-aware* manner with manual annotations to PUTs.|[CLI Usage](/docs/algorithms/ijon/algorithm_en.md#how-to-use-fuzzufs-ijon-cli)|[Algorithm Overview](/docs/algorithms/ijon/algorithm_en.md#algorithm-overview) +|IJON|Greybox|A fuzzer that can fuzz PUTs in an *internal-state-aware* manner with manual annotations to PUTs.|[CLI Usage](/docs/algorithms/ijon/algorithm_en.md#how-to-use-fuzzufs-ijon-cli)|[Algorithm Overview](/docs/algorithms/ijon/algorithm_en.md#algorithm-overview) |VUzzer|Greybox|A mutation-based fuzzer guess data structures by analyzing the PUT control flow and the data flow.|Read [Prerequisite](/docs/algorithms/vuzzer/algorithm_en.md#prerequisite) first, then [Usage on CLI](docs/algorithms/vuzzer/algorithm_en.md#usage-on-cli)|[Algorithm Overview](/docs/algorithms/vuzzer/algorithm_en.md#algorithm-overview) |libFuzzer|Greybox|CGF included in the LLVM project's compiler-rt libraries.|[How to use libFuzzer on fuzzuf](/docs/algorithms/libfuzzer/manual.md#how-to-use-libfuzzer-on-fuzzuf)|[What is libFuzzer?](/docs/algorithms/libfuzzer/algorithm_en.md#what-is-libfuzzer) |Nezha|Greybox|A fuzzer originates from libFuzzer that tries to find defects in the program by executing programs having different implementations with the same input and compares its execution results (differential fuzzing).|[How to use Nezha on fuzzuf](/docs/algorithms/nezha/manual.md#how-to-use-nezha-on-fuzzuf)|TBD From 162e4cbadb384c740562e62c5d1a34d30e1fff2c Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Thu, 3 Mar 2022 18:38:10 +0900 Subject: [PATCH 27/30] fix diff --- docs/algorithms/ijon/algorithm_en.md | 3 +++ docs/algorithms/ijon/algorithm_ja.md | 3 +++ 2 files changed, 6 insertions(+) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index aa467728..f4bd3f2a 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -67,6 +67,9 @@ index 50b1b05..aa022f6 100644 #include #include +#include + + #define compare(x,y) IJON_CTX(compare_w((x),(y))) + bool compare_w(int x, int y){ ``` For example, you can build test.c and fuzz the produced binary with the following commands: diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index 6798532f..870a446e 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -69,6 +69,9 @@ index 50b1b05..aa022f6 100644 #include #include +#include + + #define compare(x,y) IJON_CTX(compare_w((x),(y))) + bool compare_w(int x, int y){ ``` 例えば、test.cをビルドしたバイナリは、以下のようにしてファジングできます: From 98087a33b70ce813ec077e8bb8e4849cdc8c3a42 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Mon, 7 Mar 2022 09:26:47 +0900 Subject: [PATCH 28/30] Reflect review15 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index f4bd3f2a..c6cbd9e9 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -75,7 +75,7 @@ index 50b1b05..aa022f6 100644 For example, you can build test.c and fuzz the produced binary with the following commands: ```bash -$ (path_to_ijon)/llvm_mode/afl-clang-fast (path_to_ijon)/test.c -o test +$ (path_to_ijon)/afl-clang-fast (path_to_ijon)/test.c -o test $ mkdir /tmp/ijon_test_indir/ && echo hello > /tmp/ijon_test_indir/hello $ fuzzuf ijon -i /tmp/ijon_test_indir/ ./test ``` From 8574475b8e3bfaa67613131b7375c3b5b64662c2 Mon Sep 17 00:00:00 2001 From: potetisensei Date: Mon, 7 Mar 2022 09:27:26 +0900 Subject: [PATCH 29/30] Reflect review16 Co-authored-by: Yudai --- docs/algorithms/ijon/algorithm_ja.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index 870a446e..7dc74791 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -77,7 +77,7 @@ index 50b1b05..aa022f6 100644 例えば、test.cをビルドしたバイナリは、以下のようにしてファジングできます: ```bash -$ (path_to_ijon)/llvm_mode/afl-clang-fast (path_to_ijon)/test.c -o test +$ (path_to_ijon)/afl-clang-fast (path_to_ijon)/test.c -o test $ mkdir /tmp/ijon_test_indir/ && echo hello > /tmp/ijon_test_indir/hello $ fuzzuf ijon -i /tmp/ijon_test_indir/ ./test ``` From 8aa518ce8013750aa68a9a7b6ca383a7384b2787 Mon Sep 17 00:00:00 2001 From: hugeh0ge Date: Mon, 7 Mar 2022 12:27:56 +0900 Subject: [PATCH 30/30] add outcome --- docs/algorithms/ijon/algorithm_en.md | 2 +- docs/algorithms/ijon/algorithm_ja.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/algorithms/ijon/algorithm_en.md b/docs/algorithms/ijon/algorithm_en.md index c6cbd9e9..465c8741 100644 --- a/docs/algorithms/ijon/algorithm_en.md +++ b/docs/algorithms/ijon/algorithm_en.md @@ -80,7 +80,7 @@ $ mkdir /tmp/ijon_test_indir/ && echo hello > /tmp/ijon_test_indir/hello $ fuzzuf ijon -i /tmp/ijon_test_indir/ ./test ``` -Here, you don't need to specify `@@` in the last command because the binary receives inputs via stdin. +Here, you don't need to specify `@@` in the last command because the binary receives inputs via stdin. If IJON's instrumentation tool and fuzzer operate properly, the fuzzer will detect a crash within 3 to 5 minutes with a high probability. While test.c and test2.c gives you an idea how you can use annotations, you can check README and source code in IJON's repo to understand their further usage. diff --git a/docs/algorithms/ijon/algorithm_ja.md b/docs/algorithms/ijon/algorithm_ja.md index 7dc74791..e32cbbdc 100644 --- a/docs/algorithms/ijon/algorithm_ja.md +++ b/docs/algorithms/ijon/algorithm_ja.md @@ -82,7 +82,7 @@ $ mkdir /tmp/ijon_test_indir/ && echo hello > /tmp/ijon_test_indir/hello $ fuzzuf ijon -i /tmp/ijon_test_indir/ ./test ``` -test.cは標準入力から入力を受け付けるため、`@@`を指定する必要がないことに注意してください。 +test.cは標準入力から入力を受け付けるため、`@@`を指定する必要がないことに注意してください。IJONの計装ツールおよびファザーが正常に機能している場合、高い確率で3分から5分程度の間にクラッシュが発見されます。 test.cおよびtest2.cを見れば、アノテーションの付け方のイメージを掴むことができますが、 より詳しいアノテーションの使い方については、IJONのREADMEおよびソースコードを参照してください。