Improved Runtime of Function Process in process.go

[neilnaveen]

Description of the PR

Fixes: #1225

This PR addresses the performance issue raised in #1225 regarding the noticeable performance drawbacks when handling deeply nested structures in the processHelper function.

Changes:

1. Initially attempted to refactor processHelper from a recursive function to an iterative approach. However, this change did not yield significant runtime improvements.
2. Profiling revealed that the Unpack function, which utilized encoding/json, was the primary contributor to the extended runtime.
3. Switched from using encoding/json to json-iterator/go for JSON operations, resulting in a significant runtime improvement.

Benchmark Results:

Test case used for benchmarking:

sd := SimpleDoc{  
   Issuer: "google.com",  
   Info:   "this is a cool document",  
   Nested: []SimpleDoc{},  
}  
  
cur := sd  
for i := 0; i < 4000; i++ {  
   k := SimpleDoc{Issuer: "google.com", Info: fmt.Sprintf("this is a cooler nested doc %d", i), Nested: []SimpleDoc{cur}}  
  
   cur = k  
}  
// marshal the document  
blob, _ := json.Marshal(cur)  
  
// create a document  
  
overflowDoc := processor.Document{  
   Blob:   blob,  
   Type:   simpledoc.SimpleDocType,  
   Format: processor.FormatJSON,
}

• Using encoding/json: Document processed in 29.227243671s
• Using json-iterator/go: Document processed in 9.27833562s

As observed, json-iterator/go reduced the runtime by approximately two-thirds.

PR Checklist

• All commits have a Developer Certificate of Origin (DCO) -- they are generated using -s flag to git commit.
• All new changes are covered by tests
• If GraphQL schema is changed, make generate has been run
• If collectsub protobuf has been changed, make proto has been run
• All CI checks are passing (tests and formatting)
• All dependent PRs have already been merged

[pxp928]

LGTM! Thanks for the performance improvement @neilnaveen!

[pxp928]

@neilnaveen a quick rebase and this will auto merge

[neilnaveen]

@neilnaveen a quick rebase and this will auto merge

Thanks, I have fixed the conflicts.