Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal of internal cache refactor #4863

Closed
Mzack9999 opened this issue Mar 11, 2024 · 1 comment · Fixed by #4867
Closed

Proposal of internal cache refactor #4863

Mzack9999 opened this issue Mar 11, 2024 · 1 comment · Fixed by #4867
Assignees
Labels
Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.

Comments

@Mzack9999
Copy link
Member

Nuclei version:

main|dev

Current Behavior:

Potential internal cross-references confuses the gc, preventing heap structures from being correctly dismissed and recycled. Actually the OnResult reference within the template cached structure is the cause of the issue, partially mitigated by purging the cache within a specific use paradigm. The issue potentially remains and go unnoticed with future changes.
Ref #4836

Expected Behavior:

The cache should be restructured as follows:

  • Common cache between parsers and templates package, unless there is a specific reason for redundancy
  • Purgeable unique global cache containing only a generic and common template representation (no executor options, no external pointer references)
  • During execution, requests are "compiled" in their final form, which is scoped for the process duration, and should return reference-free structures (there shouldn't be reason for a Result object to hold references to internal structures, as mostly only string and maps of strings are involved)
@Mzack9999 Mzack9999 added the Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Mar 11, 2024
@Mzack9999 Mzack9999 self-assigned this Mar 11, 2024
@tarunKoyalwar
Copy link
Member

Notes

  • cache in parsers is used for initial loading of template with purpose of filtering them using TagFilters, hence all templates available are loaded and cached here to allow loading using different loading options
  • cache in templates contains compiled + valid templates that are actually loaded and will be used. the only requirement for this cache is because of workflow templates where we allow loading templates via tags

@Mzack9999 Mzack9999 changed the title Propsal of internal cache refactor Proposal of internal cache refactor Mar 12, 2024
@Mzack9999 Mzack9999 linked a pull request Mar 14, 2024 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants