Validation Performance: File level providedSymbols and requiredSymbols

[markwpearce]

benchmark: Running benchmarks:

    validate@local   --------- 271.514 ops/sec
    validate@0.65.10 --------- 101.219 ops/sec

⚡ Huge improvement to validation times! ⚡

How it works:

• When a file changes:
  • the file is broken down into individually-validatable segments (eg. each function statement, etc.)
  • each segment is indexed for symbols it requires to compile
  • all the symbols that are available externally to the file are indexed.
  • all the symbols that file requires are indexed (this is a Set of all the required symbols for all its segments)
  • all the symbols that have incompatible types to the previously-known types are marked especially.
• When there's a program validation
  • for the files that changed, check to see:
    • if there are any "duplicate symbols" in the same scope
    • if the required symbols are are compatible for each scope the file is in
    • if there's a required symbol that is not in any scope
  • do a scope validation, but also pass along a reference to files that have changed, and what the changes are
• When validating a scope:
  • Mark each diagnostic if it was generated for a full scope validation vs. a validation based on an AST segment
  • clear any previously generated diagnostics if they were on a full scope validation
  • for each file in the scope
    • validate the entire file if it's changed - but only once!
    • if it hasn't changed, and it does not require any symbols that have changed since last validation, do not validate!
    • if it does require something that has changed, than only for the segments in the file that require that symbol, validate the segment.
    • do appropriate clearing of diagnostics attached to the scope based on file and segment re-validations.

Known issues:
The language server still weirdly re-validates for all open files, and there seems to be a race condition somewhere. I see it most notably on Brighterscript files, and when using classes/types defined in namespaces** these are fixed!**

[rokucommunity-bot]

Hey there! I just built a new temporary npm package based on 48f614c. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-0.66.0-alpha.7-providers-and-requires.20231117211721.tgz

[rokucommunity-bot]

Hey there! I just built a new temporary npm package based on 1a3fa63. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-0.66.0-alpha.7-providers-and-requires.20231118025148.tgz