jq_fuzz_fixed: count-down loop times out if the input is a big number

[emanuele6]

jq_fuzz_fixed tests [.,1]|until(.[0] < 1; [.[0] - 1, .[1] * .[0]])|.[1].
If one the input is a large number e.g. 1e100, that will timeout on oss-fuzz:

$ jq '[.,1]|until(.[0] < 1; [.[0] - 1, .[1] * .[0]])|.[1]' <<<1e100

Should we just remove that program from the list?

Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64899

[wader]

I wonder how similar projects handle these kinds of cases?

[emanuele6]

cc @DavidKorczynski
Any suggestions?

[DavidKorczynski]

I think for now I'd probably just leave a comment in https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64899 saying e.g. "This timeout is valid and just takes a long time to compute. Wont fix". Then I can mark them as resolved.

If there are to many timeouts occurring so it becomes an annoyance then we can adjust the fuzzers?

[wader]

@DavidKorczynski 👍 does oss-fuzz have some limit on how many things it report per day etc? thinking if the timeouts might stop it from fuzzing?

[emanuele6]

I thought it would be problematic since this can happen very easily, any large number running that test will trigger it, but maybe not.

[emanuele6]

P.S. If I add something like this in jq_fuzz_fixed, will I be able to see it in the "detailed report" log on the oss-fuzz page?

diff --git a/tests/jq_fuzz_fixed.cpp b/tests/jq_fuzz_fixed.cpp
index 163eef4..88106c9 100644
--- a/tests/jq_fuzz_fixed.cpp
+++ b/tests/jq_fuzz_fixed.cpp
@@ -1,4 +1,5 @@
 #include <fuzzer/FuzzedDataProvider.h>
+#include <iostream>
 #include <string>
 
 #include "jq.h"
@@ -267,13 +268,15 @@ extern "C" int LLVMFuzzerTestOneInput(uint8_t *data, size_t size) {
 
   int idx = fdp.ConsumeIntegralInRange<int>(
       0, (sizeof(jq_progs) / sizeof(char *)) - 1);
+  const char *jq_prog = jq_progs[idx];
+  std::cerr << "jq_progs[" << idx << "]: " << jq_prog << std::endl;
 
   jq_state *jq = NULL;
   jq = jq_init();
   if (jq != NULL) {
     jq_set_attr(jq, jv_string("JQ_ORIGIN"), jv_string("/tmp/"));
 
-    if (jq_compile(jq, jq_progs[idx])) {
+    if (jq_compile(jq, jq_prog) {
       // Process to jv_parse and then jv_next
       jv input = jv_parse(parse_payload1.c_str());
       if (jv_is_valid(input)) {

Currently, I have just been running all the tests manually with:

jq -r '.[] | @sh "./jq -- \(.) <<< theinput # /orlessthanpathtofile"' teststhing | PS4=$ bash -x

(teststhing is adapted from the c++ source file and contains a JSON array with all the jq_progs.)

Until I reach the one that triggers problems, because I have not figured out an easy way to tell which program was used from just the seed.

[DavidKorczynski]

Until I reach the one that triggers problems, because I have not figured out an easy way to tell which program was used from just the seed.

In order to make it easy to see which program was used I'd suggest converting

jq/tests/jq_fuzz_fixed.cpp

Lines 8 to 29 in 2ccab04

	const char *jq_progs[] = {
	". / \", \"",
	".[]",
	"$ENV.PAGER",
	".[0]",
	". < 0.12345678901234567890123456788",
	".[] == 1",
	".[] | (1 / .)?",
	"10 / . * 3",
	"[1,2,empty,3]",
	"1, empty, 2",
	"[.,1]|until(.[0] < 1; [.[0] - 1, .[1] * .[0]])|.[1]",
	".[-2:]",
	".[-2]",
	".[2]",
	"[ .[] | . * 2]",
	".[2:4]",
	"(. + 2) * 5",
	".[:3]",
	".[4,2]",
	"42 and \"a string\"",
	"4 - .a",

into a large switch statement as opposed to array indexing -- in that we way will have line numbers in stack traces which will make it clear which program triggered a certain issues.

[wader]

That would be great. I've managed to reproduce using the instructions at https://google.github.io/oss-fuzz/advanced-topics/reproducing/ (had to add explicit architecture flag in some place) but being to see query and be able to possibly reproduce without the oss-fuzz setup would be nice

[DavidKorczynski]

@DavidKorczynski 👍 does oss-fuzz have some limit on how many things it report per day etc? thinking if the timeouts might stop it from fuzzing?

No limit as such. If the timeouts are really easy to trigger and it becomes spammy then I think our best option is to adjust the fuzzer.

[emanuele6]

Thank you