Skip to content

Commit

Permalink
Update benchmarks; run tests on Go 1.19; bump min version to 1.15
Browse files Browse the repository at this point in the history
Benchmarks between go1.18.1 (old/orig) and go1.19 (new/goawk) below.
Microbenchmarks are a bit all over the place, but 1.5% faster overall,
and macro-benchmarks are 3.5% faster overall.

name                    old time/op  new time/op   delta
IncrDecr-8               135ns ± 1%    160ns ± 1%  +18.38%  (p=0.008 n=5+5)
ArrayOperations-8        539ns ± 1%    603ns ± 1%  +11.93%  (p=0.008 n=5+5)
RepeatNew-8             95.1ns ± 2%  103.5ns ± 2%   +8.88%  (p=0.008 n=5+5)
IfStatement-8            150ns ± 1%    157ns ± 1%   +4.81%  (p=0.008 n=5+5)
ConcatLarge-8           1.59ms ± 2%   1.66ms ± 1%   +4.58%  (p=0.008 n=5+5)
CondExpr-8               150ns ± 1%    157ns ± 0%   +4.33%  (p=0.008 n=5+5)
ForInLoop-8             1.68µs ± 2%   1.74µs ± 3%   +3.31%  (p=0.032 n=5+5)
BinaryOperators-8        401ns ± 0%    412ns ± 1%   +2.73%  (p=0.016 n=4+5)
BuiltinIndex-8          1.00µs ± 1%   0.99µs ± 2%     ~     (p=0.548 n=5+5)
BuiltinSplitSpace-8     35.2µs ± 1%   35.7µs ± 1%     ~     (p=0.056 n=5+5)
BuiltinSplitRegex-8     89.6µs ± 2%   90.1µs ± 4%     ~     (p=1.000 n=5+5)
BuiltinSub-8            14.0µs ± 2%   13.9µs ± 4%     ~     (p=0.421 n=5+5)
BuiltinSubAmpersand-8   14.0µs ± 2%   13.8µs ± 1%     ~     (p=0.095 n=5+5)
BuiltinGsub-8           17.0µs ± 3%   16.6µs ± 2%     ~     (p=0.095 n=5+5)
BuiltinGsubAmpersand-8  16.6µs ± 2%   16.8µs ± 4%     ~     (p=1.000 n=5+5)
RecursiveFunc-8         14.5µs ± 2%   14.2µs ± 3%     ~     (p=0.095 n=5+5)
SimplePattern-8          136ns ± 2%    135ns ± 1%     ~     (p=0.310 n=5+5)
GetField-8               327ns ± 3%    325ns ± 2%     ~     (p=0.841 n=5+5)
ConcatSmall-8            349ns ± 2%    347ns ± 3%     ~     (p=1.000 n=5+5)
Printf-8                4.45µs ± 3%   4.40µs ± 2%     ~     (p=0.151 n=5+5)
RepeatIONew-8           1.08µs ± 1%   1.07µs ± 1%     ~     (p=0.063 n=5+5)
CSVInputGoAWK-8          395ns ± 5%    394ns ± 4%     ~     (p=0.548 n=5+5)
CSVInputReader-8         284ns ± 8%    288ns ± 4%     ~     (p=1.000 n=5+5)
CSVOutputGoAWK-8         371ns ± 4%    375ns ± 1%     ~     (p=0.222 n=5+5)
CSVOutputWriter-8        237ns ± 6%    235ns ± 7%     ~     (p=0.841 n=5+5)
RepeatIOExecProgram-8   24.3µs ± 0%   23.8µs ± 1%   -1.89%  (p=0.008 n=5+5)
SetField-8               827ns ± 3%    809ns ± 1%   -2.17%  (p=0.032 n=5+5)
BuiltinSprintf-8        8.01µs ± 1%   7.84µs ± 2%   -2.19%  (p=0.032 n=5+5)
RepeatExecProgram-8     13.9µs ± 1%   13.5µs ± 2%   -2.42%  (p=0.016 n=5+5)
LocalVars-8              277ns ± 1%    268ns ± 2%   -3.06%  (p=0.008 n=5+5)
NativeFunc-8            10.3µs ± 1%   10.0µs ± 1%   -3.35%  (p=0.008 n=5+5)
BuiltinSubstr-8          744ns ± 1%    710ns ± 2%   -4.64%  (p=0.008 n=5+5)
FuncCall-8              2.72µs ± 4%   2.59µs ± 2%   -4.92%  (p=0.008 n=5+5)
AugAssign-8              992ns ± 2%    942ns ± 1%   -4.97%  (p=0.008 n=5+5)
Print-8                 2.29µs ± 0%   2.18µs ± 1%   -4.98%  (p=0.008 n=5+5)
BuiltinMatch-8          2.66µs ± 3%   2.50µs ± 1%   -6.32%  (p=0.008 n=5+5)
ForLoop-8               2.41µs ± 1%   2.24µs ± 3%   -7.23%  (p=0.008 n=5+5)
Comparisons-8            322ns ± 2%    298ns ± 2%   -7.66%  (p=0.008 n=5+5)
RegexMatch-8            1.08µs ± 1%   0.99µs ± 1%   -8.39%  (p=0.008 n=5+5)
SimpleBuiltins-8         734ns ± 2%    672ns ± 1%   -8.49%  (p=0.008 n=5+5)
BuiltinLength-8          430ns ± 2%    391ns ± 1%   -8.91%  (p=0.008 n=5+5)
ConcatTwo-8             98.3ns ± 2%   89.0ns ± 1%   -9.44%  (p=0.008 n=5+5)
Assign-8                 275ns ± 1%    247ns ± 1%  -10.21%  (p=0.008 n=5+5)
GlobalVars-8             281ns ± 1%    252ns ± 3%  -10.32%  (p=0.008 n=5+5)
[Geo mean]              1.50µs        1.47µs        -1.52%

Test                         |    goawk |     orig |      awk |     gawk |     mawk
---------------------------- | -------- | -------- | -------- | -------- | --------
tt.01 (print)                |     2.06 |     1.53 |     1.00 |     1.84 |     2.41
tt.02 (print NR NF)          |     1.66 |     1.57 |     1.00 |     1.79 |     1.86
tt.02a (print length)        |     1.40 |     1.46 |     1.00 |     1.68 |     1.96
tt.03 (sum length)           |     1.30 |     1.32 |     1.00 |     3.91 |     1.84
tt.03a (sum field)           |     1.22 |     1.26 |     1.00 |     4.22 |     1.80
tt.04 (printf fields)        |     1.05 |     1.05 |     1.00 |     1.69 |     2.63
tt.05 (concat fields)        |     1.13 |     1.11 |     1.00 |     1.57 |     2.16
tt.06 (count lengths)        |     1.42 |     1.41 |     1.00 |     2.61 |     1.99
tt.07 (even fields)          |     1.18 |     1.12 |     1.00 |     1.52 |     1.60
tt.08 (even lengths)         |     1.95 |     1.87 |     1.00 |     1.14 |     2.64
tt.08z (regex simple)        |     3.27 |     3.24 |     1.00 |     2.81 |     6.27
tt.09 (regex starts with)    |     2.37 |     2.44 |     1.00 |     2.49 |     5.97
tt.10 (regex ends with)      |     0.38 |     0.37 |     1.00 |     1.36 |     3.43
tt.10a (regex ends with var) |     0.35 |     0.34 |     1.00 |     1.38 |     1.35
tt.11 (substr)               |     2.93 |     2.68 |     1.00 |     1.26 |     4.15
tt.12 (update fields)        |     1.28 |     1.20 |     1.00 |     1.58 |     1.55
tt.13 (array ops)            |     4.01 |     3.91 |     1.00 |     3.07 |     6.00
tt.13a (array printf)        |     2.73 |     2.72 |     1.00 |     2.10 |     4.74
tt.14 (function call)        |     1.24 |     1.25 |     1.00 |     0.60 |     1.61
tt.15 (format lines)         |     0.72 |     0.69 |     1.00 |     0.97 |     2.36
tt.16 (count words)          |     1.58 |     1.57 |     1.00 |     1.28 |     2.09
tt.big (complex program)     |     1.88 |     1.90 |     1.00 |     2.11 |     3.72
tt.x1 (mandelbrot)           |     2.23 |     2.23 |     1.00 |     1.32 |     3.44
tt.x2 (sum loop)             |     1.81 |     1.71 |     1.00 |     1.07 |     2.64
---------------------------- | -------- | -------- | -------- | -------- | --------
**Geo mean**                 | **1.49** | **1.44** | **1.00** | **1.71** | **2.63**
  • Loading branch information
benhoyt committed Aug 3, 2022
1 parent 667dd75 commit 41beaa4
Show file tree
Hide file tree
Showing 4 changed files with 6 additions and 6 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:

strategy:
matrix:
go: ['1.18', '1.17', '1.16', '1.15', '1.14']
go: ['1.19', '1.18', '1.17', '1.16', '1.15']

name: Go ${{ matrix.go }} on Linux

Expand All @@ -34,7 +34,7 @@ jobs:

strategy:
matrix:
go: ['1.18', '1.14']
go: ['1.19', '1.15']

name: Go ${{ matrix.go }} on Windows

Expand All @@ -59,7 +59,7 @@ jobs:

strategy:
matrix:
go: ['1.18', '1.14']
go: ['1.19', '1.15']

name: Go ${{ matrix.go }} on macOS

Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module github.com/benhoyt/goawk

go 1.14
go 1.15
2 changes: 1 addition & 1 deletion scripts/fuzz_input.sh
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
#!/bin/sh
go1.18rc1 test ./interp -run=^$ -fuzz=Input -parallel=4
go test ./interp -run=^$ -fuzz=Input -parallel=4
2 changes: 1 addition & 1 deletion scripts/fuzz_source.sh
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
#!/bin/sh
go1.18rc1 test ./interp -run=^$ -fuzz=Source -parallel=4
go test ./interp -run=^$ -fuzz=Source -parallel=4

0 comments on commit 41beaa4

Please sign in to comment.