Transparent Memoization via func Annotation

[Mzack9999]

Proposed changes

This PR implements a transparent memoization mechanism via functions annotations and AST parsing, requiring no changes within the codebase other than replacing package.Function(xx) with memo.Function(xx) (memo is the newly generated package with memoized functions).
A function to memoize should be annotated in the following way:

// file test.go
package test

// @memo
func A(xx, yy int) int {
    // heavy workload
   return complexResult
}

The new utility parses the go files and within a new destination package creates corresponding files exposing the same functions signature, but wrapped within a memoization mechanism:

• Functions without params and with/without return values are wrapped around sync.Once that automatically declares local variables to store arbitrary return values, capture in-flight requests and return the result
• Functions with params works in a similar manner, by caching a local struct that works as result values holder and capturing in-flight requests through a unique hash calculated as a combination of function name, and parameter values

Pros:

• Adaptive runtime lazy clustering via function hashing with single flight request of stateless functions yielding same output
• Traffic reduction of 10X

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Example

Generation is not automated via GH action on purpose as helpers should be fully stateless and intentionally marked by developer.
Command:

# generate memoized functions from annotated // @memo
$ go run . -src "/Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs"
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/mssql/mssql.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/mysql/mysql.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/mysql/mysql_private.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/oracle/oracle.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/pop3/pop3.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/postgres/postgres.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/rdp/rdp.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/redis/redis.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/rsync/rsync.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/smb/smb.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/smb/smb_private.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/smb/smbghost.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/smtp/smtp.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/ssh/ssh.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/telnet/telnet.go
2024/02/20 14:19:04 processing: /Users/marcornvh/go/src/github.com/projectdiscovery/nuclei/pkg/js/libs/vnc/vnc.go

$ echo 192.168.5.45 | nuclei -duc -verbose -rl 0 -t /Users/marcornvh/nuclei-templates/javascript/ -debug

Before:

After:

[tarunKoyalwar]

lgtm !

i think all of known independent heavy functions are memoized. with scan logs we will be able to identify and memoize more of them