Skip to content

[1.8.0] Performance Issue on large array update. #3345

@zogwarg

Description

@zogwarg

What

When running updates on large slices of large arrays. jq-1.8.0 is much slower than jq-1.7.1
One visible change is a LOADV command being used in 1.8.0 where LOADVN was used in 1.7.1

ENV: Specifically tested for the darwin-arm64 binaries but probably present in other ones.

Reproducing

Minimal reproducing, but the LOADV vs LOADVN discrepancy is present in many cases when updating arrays.

  1. Download the binaries to be run locally,
  2. ( /usr/bin/time -po /dev/stdout jq-1.7.1 -n '30000 as $size | [range($size)] | getpath(range(100;$size-100) | [.]) |= 0 | empty' --debug-dump-disasm ) > debug.1.7.1.txt
  3. ( /usr/bin/time -po /dev/stdout jq-1.8.0 -n '30000 as $size | [range($size)] | getpath(range(100;$size-100) | [.]) |= 0 | empty' --debug-dump-disasm ) > debug.1.8.0.txt

Observered diff:

diff --git a/debug.1.7.1.txt b/debug.1.8.0.txt
index 2bde2f6..2c13366 100644
--- a/debug.1.7.1.txt
+++ b/debug.1.8.0.txt
@@ -52,7 +52,7 @@ _modify:3:
   0027 DUPN
   0028 CALL_JQ path:1^1 @lambda:0
   0034 STOREV $p:2
-  0037 LOADVN $reduce:1
+  0037 LOADV $reduce:1
   0040 DUP
   0041 DUP
   0042 POP
@@ -203,6 +203,6 @@ range:4:
   0000 LOADK 0
   0002 RET

-real 0.07
-user 0.06
+real 2.91
+user 2.90
 sys 0.00

Noticeable speed drop for size 30000, 0.07s to 2.91s, compounded the larger the array.

Expected behavior

  • Keep fast good performance on large arrays slices updates.
  • Provide a way to force the use of LOADVN when possible.
  • and/or documentation update about performance tradeoffs.

Additional Question

Is there any way to rum the dissasembled code with the new binary to see if it would remain correct, and actually complete quickly?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions