ci: Add services fuzz test for read/write/range_read #2710

dqhl76 · 2023-07-25T14:55:09Z

part of: #2551

Signed-off-by: dqhl76 <dqhl76@gmail.com>

.github/workflows/fuzz_test.yml

Signed-off-by: dqhl76 <dqhl76@gmail.com>

dqhl76 · 2023-07-26T02:05:01Z

Would you like to take a look again? /cc @Xuanwo @oowl

I use matrix strategy to split them into 9 jobs ([reader, range_reader, writer] * [fs, memory, s3])

Advantages:

It is easier to check the output log

Disadvantages:

added 9 jobs, more complex
time may still too long (9 mins)

.github/workflows/fuzz_test.yml

Signed-off-by: dqhl76 <dqhl76@gmail.com>

dqhl76 · 2023-07-26T05:42:28Z

All comment solved. 14581f4

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Xuanwo · 2023-07-26T06:14:32Z

wow, seems we catch a case!

==1660== ERROR: libFuzzer: deadly signal
    #0 0x556526ec0d11  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2c4bd11) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #1 0x55652c048c7e  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7dd3c7e) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #2 0x55652c0272d9  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7db22d9) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #3 0x7f0c9a6e451f  (/lib/x86_64-linux-gnu/libc.so.6+0x4251f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #4 0x7f0c9a738a7b  (/lib/x86_64-linux-gnu/libc.so.6+0x96a7b) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #5 0x7f0c9a6e4475  (/lib/x86_64-linux-gnu/libc.so.6+0x42475) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #6 0x7f0c9a6ca7f2  (/lib/x86_64-linux-gnu/libc.so.6+0x287f2) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #7 0x55652c139436  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7ec4436) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #8 0x556526e1ab36  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2ba5b36) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #9 0x55652c00bc34  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7d96c34) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #10 0x55652c12d77f  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7eb877f) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #11 0x55652c12d4c0  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7eb84c0) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #12 0x55652c12a545  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7eb5545) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #13 0x55652c12d251  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7eb8251) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #14 0x556526e1e1e2  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2ba91e2) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #15 0x556526e1e272  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2ba9272) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #16 0x556527064af1  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2defaf1) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #17 0x556526f[365](https://github.com/apache/incubator-opendal/actions/runs/5665172769/job/15349790214?pr=2710#step:4:366)8a  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2cc158a) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #18 0x556527eb37f9  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x3c3e7f9) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #19 0x5565273116d8  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x309c6d8) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #20 0x556527315ccf  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x30a0ccf) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #21 0x5565271bf7db  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2f4a7db) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #22 0x5565271c2462  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2f4d462) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #23 0x5565271c4409  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2f4f409) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #24 0x556527315673  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x30a0673) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #25 0x556527319154  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x30a4154) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #26 0x556527317807  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x30a2807) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #27 0x55652c006aa8  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7d91aa8) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #28 0x55652c00be47  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7d96e47) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #29 0x55652c00b0a5  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7d960a5) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #30 0x55652c027825  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7db2825) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #31 0x55652c02cc43  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7db7c43) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #32 0x55652c02dc38  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7db8c38) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #33 0x55652c030027  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7dbb027) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #34 0x55652c01ab5f  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x7da5b5f) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #35 0x556526e1eab6  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2ba9ab6) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)
    #36 0x7f0c9a6cbd8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #37 0x7f0c9a6cbe3f  (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #38 0x556526e1ec54  (/home/runner/work/incubator-opendal/incubator-opendal/target/fuzz_range_reader+0x2ba9c54) (BuildId: 21214199866a4da26e035c5fce3308b35dfdd5ff)

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 3 ChangeBit-CopyPart-InsertRepeatedBytes-; base unit: 260e69d75c55db9fab30c183db2e8737e88ac072
0x0,0x0,0x1,0x0,0x8,0x1,0x0,0x8,0x8,0x0,0xaf,0xaf,0xaf,0xaf,0xaf,0xaf,0xaf,0x93,0x8,0x0,0x93,
\000\000\001\000\010\001\000\010\010\000\257\257\257\257\257\257\257\223\010\000\223
artifact_prefix='./'; Test unit written to ./crash-b80f[377](https://github.com/apache/incubator-opendal/actions/runs/5665172769/job/15349790214?pr=2710#step:4:378)b3b78e8ad25adc9a7501c973af3ca7d1d
Base64: AAABAAgBAAgIAK+vr6+vr6+TCACT

How can we reproduce this case?

BTW, let's create a new issue for this. We can merge this PR first!

dqhl76 · 2023-07-26T06:17:07Z

How can we reproduce this case?

I think I may need to setup the environment variable to reproduce the case. Let me check out

Xuanwo · 2023-07-26T06:18:07Z

I think I may need to setup the environment variable to reproduce the case. Let me check out

Thanks a lot! I feel like we can remove our fuzz_xxx test in behaviour test in the future.

Xuanwo · 2023-07-26T06:38:23Z

Maybe we should upload the crash-b80f377b3b78e8ad25adc9a7501c973af3ca7d1d files to artifacts if our test failed. So we can use cargo fuzz run crash-xxx to re-pro it?

dqhl76 · 2023-07-26T06:43:44Z

Maybe we should upload the crash-b80f377b3b78e8ad25adc9a7501c973af3ca7d1d files to artifacts if our test failed.

Yes. I reproduce it by using the crash file successfully. However, I am trying to find a way to print the crash input in the log. When I use cargo fuzz run .. it will print, but if it run by a build artifact seems not.

Xuanwo · 2023-07-26T06:46:16Z

However, I am trying to find a way to print the crash input in the log.

It's ok to save the crash file directly. Printing the crash input in the log doesn't help with our debugging: it's hard for us to implement the same test case. Converting a fuzz input into unit test is not a simple thing (I guess).

dqhl76 · 2023-07-26T06:47:09Z

Ok. Let's upload the crash file

Signed-off-by: dqhl76 <dqhl76@gmail.com>

.github/workflows/fuzz_test.yml

Signed-off-by: dqhl76 <dqhl76@gmail.com>

dqhl76 · 2023-07-26T07:20:08Z

we got the crash file fuzz_crashes

Reproduce by: cargo fuzz run fuzz_range_reader /crash-a43d017086414fd4425807171257611ddfcc3122

Get the Debug:fmt of input: cargo fuzz fmt fuzz_range_reader /crash-a43d017086414fd4425807171257611ddfcc3122

.github/workflows/fuzz_test.yml

Xuanwo · 2023-07-26T07:21:55Z

we got the crash file fuzz_crashes

Reproduce by: cargo fuzz run fuzz_range_reader /crash-a43d017086414fd4425807171257611ddfcc3122

Get the Debug:fmt of input: cargo fuzz fmt fuzz_range_reader /crash-a43d017086414fd4425807171257611ddfcc3122

Thanks a lot! We can add them in the fuzz/README.md.

Signed-off-by: dqhl76 <dqhl76@gmail.com>

.github/workflows/fuzz_test.yml

Signed-off-by: dqhl76 <dqhl76@gmail.com>

dqhl76 · 2023-07-26T08:53:14Z

crash_fs_fuzz_range_reader_refs/pull/2710/merge_1_05cd2fb42732366e47a32e84f7a2fca96d7b340b
I think this crash file name has enough information and can avoid conflict

Oh, no.
fuzz-test-run-fs (fuzz_range_reader)

Artifact name is not valid: crash_fs_fuzz_range_reader_refs/pull/2710/merge_1_05cd2fb42732366e47a32e84f7a2fca96d7b340b. Contains the following character:  Forward slash /
          
Invalid characters include:  Double quote ", Colon :, Less than <, Greater than >, Vertical bar |, Asterisk *, Question mark ?, Carriage return \r, Line feed \n, Backslash \, Forward slash /
          
These characters are not allowed in the artifact name due to limitations with certain file systems such as NTFS. To maintain file system agnostic behavior, these characters are intentionally not allowed to prevent potential problems with downloads on different file systems.

I will fix this in the evening.

Signed-off-by: dqhl76 <dqhl76@gmail.com>

.github/workflows/fuzz_test.yml

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Xuanwo · 2023-07-26T10:10:47Z

Wait for #2718

dqhl76 added 2 commits July 25, 2023 22:53

ci: add fuzz test

e2f5b30

Signed-off-by: dqhl76 <dqhl76@gmail.com>

fix: change time limit

57c3700

Signed-off-by: dqhl76 <dqhl76@gmail.com>

dqhl76 requested review from Xuanwo and PsiACE as code owners July 25, 2023 14:55

github-actions bot added the releases-note/ci label Jul 25, 2023

fix: fmt

bfe9621

Signed-off-by: dqhl76 <dqhl76@gmail.com>

oowl approved these changes Jul 25, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

oowl changed the title ~~ci: add fuzz test~~ ci: add s3 service fuzz test Jul 25, 2023

dqhl76 commented Jul 25, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

dqhl76 added 3 commits July 26, 2023 09:12

feat: add matrix strategy

df9e124

Signed-off-by: dqhl76 <dqhl76@gmail.com>

feat: split s3

c55332e

Signed-off-by: dqhl76 <dqhl76@gmail.com>

fix: change permission

7e49726

Signed-off-by: dqhl76 <dqhl76@gmail.com>

dqhl76 force-pushed the fuzz-ci-test branch from 4c2c685 to 7e49726 Compare July 26, 2023 01:52

Xuanwo reviewed Jul 26, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

Xuanwo reviewed Jul 26, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

fix: add release flag and make code readable

14581f4

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Xuanwo changed the title ~~ci: add s3 service fuzz test~~ ci: Add services fuzz test for read/write/range_read Jul 26, 2023

fix: fix wrong path

7ddf5d1

Signed-off-by: dqhl76 <dqhl76@gmail.com>

dqhl76 closed this Jul 26, 2023

dqhl76 reopened this Jul 26, 2023

fix: fix wrong path

6312813

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Xuanwo reviewed Jul 26, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Show resolved Hide resolved

fix: add if failure condition

23fe03a

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Xuanwo reviewed Jul 26, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

dqhl76 added 3 commits July 26, 2023 15:41

fix: add crash info to avoid conflict

71c3385

Signed-off-by: dqhl76 <dqhl76@gmail.com>

fix: add crash time to avoid conflict

ef75a2b

Signed-off-by: dqhl76 <dqhl76@gmail.com>

fix: add crash time to avoid conflict

3b6ca3b

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Xuanwo reviewed Jul 26, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

fix: add crash time to avoid conflict

c15a23b

Signed-off-by: dqhl76 <dqhl76@gmail.com>

fix: correct char

ee09437

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Xuanwo reviewed Jul 26, 2023

View reviewed changes

.github/workflows/fuzz_test.yml Outdated Show resolved Hide resolved

fix: correct char

e9e1940

Signed-off-by: dqhl76 <dqhl76@gmail.com>

Merge branch 'main' into fuzz-ci-test

e207c11

PsiACE approved these changes Jul 26, 2023

View reviewed changes

dqhl76 mentioned this pull request Jul 24, 2023

Tracking issue of implementing oss-fuzz for reader and writer #2551

Closed

3 tasks

Xuanwo merged commit 1057533 into apache:main Jul 26, 2023
34 checks passed

dqhl76 deleted the fuzz-ci-test branch July 27, 2023 02:00

oowl mentioned this pull request Jul 28, 2023

Bump to version 0.39.0 #2727

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: Add services fuzz test for read/write/range_read #2710

ci: Add services fuzz test for read/write/range_read #2710

dqhl76 commented Jul 25, 2023 •

edited

dqhl76 commented Jul 26, 2023

dqhl76 commented Jul 26, 2023 •

edited

Xuanwo commented Jul 26, 2023 •

edited

dqhl76 commented Jul 26, 2023 •

edited

Xuanwo commented Jul 26, 2023

Xuanwo commented Jul 26, 2023 •

edited

dqhl76 commented Jul 26, 2023

Xuanwo commented Jul 26, 2023 •

edited

dqhl76 commented Jul 26, 2023 •

edited

dqhl76 commented Jul 26, 2023

Xuanwo commented Jul 26, 2023

dqhl76 commented Jul 26, 2023 •

edited

Xuanwo commented Jul 26, 2023

ci: Add services fuzz test for read/write/range_read #2710

ci: Add services fuzz test for read/write/range_read #2710

Conversation

dqhl76 commented Jul 25, 2023 • edited

dqhl76 commented Jul 26, 2023

dqhl76 commented Jul 26, 2023 • edited

Xuanwo commented Jul 26, 2023 • edited

dqhl76 commented Jul 26, 2023 • edited

Xuanwo commented Jul 26, 2023

Xuanwo commented Jul 26, 2023 • edited

dqhl76 commented Jul 26, 2023

Xuanwo commented Jul 26, 2023 • edited

dqhl76 commented Jul 26, 2023 • edited

dqhl76 commented Jul 26, 2023

Xuanwo commented Jul 26, 2023

dqhl76 commented Jul 26, 2023 • edited

Xuanwo commented Jul 26, 2023

dqhl76 commented Jul 25, 2023 •

edited

dqhl76 commented Jul 26, 2023 •

edited

Xuanwo commented Jul 26, 2023 •

edited

dqhl76 commented Jul 26, 2023 •

edited

Xuanwo commented Jul 26, 2023 •

edited

Xuanwo commented Jul 26, 2023 •

edited

dqhl76 commented Jul 26, 2023 •

edited

dqhl76 commented Jul 26, 2023 •

edited