stdJSON parse performance

[sancarn]

Current benchmarks

Parsing

Parsing is probably one of the most important metrics. Generally people parse json more than they do make it. So this is an important metric.

Test code

Const cmax As Long = 100
With stdJSON.Create()
  For i = 1 To 100
    With .AddObject("x" & i)
      .Add "x", "x"
      .Add "y", 100
      .Add "longString", String$(255, "x")
      With .AddArray("test")
        .Add 1
        .Add 2
        .Add 3
      End With
    End With
  Next
  Dim s As String: s = .ToString()
End With

With stdPerformance.Measure("JsonConverter", cmax)
  For i = 1 To cmax
    Call JsonConverter.ParseJson(s)
  Next
End With
With stdPerformance.Measure("JsonBag", cmax)
  For i = 1 To cmax
    Dim x As JsonBag
    Set x = New JsonBag
    x.JSON = s
  Next
End With
With stdPerformance.Measure("stdJSON", cmax)
  For i = 1 To cmax
    Call stdJSON.CreateFromString(s)
  Next
End With

Results

JsonConverter: 5641 ms (56410µs per operation)
JsonBag: 2891 ms (28910µs per operation)
stdJSON: 7234 ms (72340µs per operation)

From these results we can see that stdJSON is significantly slower at the moment than Either Tim Hall's JSON module and JsonBag.

I think JsonBag is likely getting the performance it does by using 2 collections to store it's children data? Though this is unconfirmed. This is something that needs improving in stdJSON!

Next steps

My anticipation is that if we migrate stdJSON to use a in-class Array and HashTable (instead of IDispatch based Dictionary) this will be significantly faster than the current implementation. I.E.

Type TChild
  key as string
  val as variant
  next as long
  prev as long
End Type
Type TThis
  hashes(0 to 255) as Long
  children as TChild
End Type

Stringification

For completeness, stringification is less important but is still a useful benchmark.

Test code

Const cmax As Long = 10
  With stdJSON.Create()
    For i = 1 To 1000
      With .AddObject("x" & i)
        .Add "x", "x"
        .Add "y", 100
        .Add "longString", String$(255, "x")
        With .AddArray("test")
          .Add 1
          .Add 2
          .Add 3
        End With
      End With
    Next
    Dim s As String: s = .ToString()
  End With
  
  With stdPerformance.Optimise
    Dim timHallJSON As Object: Set timHallJSON = JsonConverter.ParseJson(s)
    Dim jsBag As New JsonBag: jsBag.JSON = s
    Dim std As stdJSON: Set std = stdJSON.CreateFromString(s)
    Dim x As String
    With stdPerformance.Measure("JsonConverter", cmax)
      For i = 1 To cmax
        x = JsonConverter.ConvertToJson(timHallJSON)
      Next
    End With
    With stdPerformance.Measure("JsonBag", cmax)
      For i = 1 To cmax
        x = jsBag.JSON
      Next
    End With
    With stdPerformance.Measure("stdJSON", cmax)
      For i = 1 To cmax
        x = std.ToString
      Next
    End With
  End With

Results

JsonConverter: 4328 ms (432800µs per operation)
JsonBag: 10578 ms (1057800µs per operation)
stdJSON: 422 ms (42200µs per operation)

As we can see stdJSON is much better than the competition here, so at least we have something right. I think this is largely because both JSONConverter and JsonBag commonly use str = left(str, len(str)-1) to handle trailing commas, which when dealing with huge strings is incredibly slow! Instead we prevent the trailing commas from being in the string in the first place. Perhaps a better solution however is leave the trailing commas alone, and trim our buffer by subtracting 1 from the buffer index after the last operation...

Next steps

Consider allowing trailing commas, and removing the commas just by subtracting 1 from the buffer index.