Add lookup tables to DiagnosticsManager to speed up diagnostic clearing by markwpearce · Pull Request #1411 · rokucommunity/brighterscript

markwpearce · 2025-02-13T16:05:14Z

TBH, I'm not sure if this will speed up validation. However, there are many times in the validation process when the Diagnostic Manager needs to clear all diagnostics based on scope, file, tag, etc. Previously, this was done by looping all diagnostics, so if there are lots, it probably went slower and slower. This change should make it much more quick to find the diagnostics that need to be cleared.

Also added some more fine-grained Scope Validation metrics that are printed out with log-level debug:

[02:25:25:633 PM] [prj0] Validating scope components/search/SearchRow.xml
[02:25:25:634 PM] [prj0] Validating scope components/search/SearchRow.xml finished. (1.46ms)
[02:25:25:634 PM] [prj0] components/search/SearchRow.xml segment metrics:
[02:25:25:634 PM] [prj0]  -  components/search/SearchRow.brs 11
[02:25:25:634 PM] [prj0] components/search/SearchRow.xml total segments validated 11
[02:25:25:634 PM] [prj0] Validation Metrics (Scope: components/search/SearchRow.xml): fileWalkTime=0.953ms, flagDuplicateFunctionTime=0.21ms, classValidationTime=0.13ms, scriptImportValidationTime=0.40ms, xmlValidationTime=0.17ms
[02:25:25:634 PM] [prj0] BscPlugin onScopeValidate finished. (1.72ms)

…e,tag,segment,etc, instead of looping all diagnostics

rokucommunity-bot · 2025-02-13T16:07:54Z

Hey there! I just built a new temporary npm package based on f77c753. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250213160536.tgz

rokucommunity-bot · 2025-02-14T14:24:36Z

Hey there! I just built a new temporary npm package based on 55e91ba. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250214142216.tgz

rokucommunity-bot · 2025-02-14T15:43:56Z

Hey there! I just built a new temporary npm package based on 2be68e9. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250214154143.tgz

rokucommunity-bot · 2025-02-20T17:47:51Z

Hey there! I just built a new temporary npm package based on dd4c33b. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250220174529.tgz

…ovidedTreeMap

rokucommunity-bot · 2025-02-20T18:15:01Z

Hey there! I just built a new temporary npm package based on be504ae. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250220181209.tgz

markwpearce · 2025-02-20T18:18:43Z

-
-                    clonedNsContainer.symbolTable.mergeSymbolTable(nsContainer.symbolTable);
-                    namespaceLookup.set(lowerNamespaceName, clonedNsContainer);
-                } else {


eliminates a bunch of empty/useless maps

rokucommunity-bot · 2025-02-20T18:19:28Z

Hey there! I just built a new temporary npm package based on c072b43. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250220181710.tgz

markwpearce · 2025-02-20T18:20:07Z

        }
        this.linkSymbolTableDisposables = [];
+
+        this.cache.delete('namespaceLookup');


The namespaceLookup was stored in the cache, and not needed outside of when the scope was linked... deleting it here makes sense

markwpearce · 2025-02-20T18:25:42Z

Some of the work here is to make the overall memory footprint smaller:

Here's before & after snapshots of the heap memory when validating & building Jellyfin (with an extra file that adds 4000+ namespaces):

rokucommunity-bot · 2025-02-20T20:04:30Z

Hey there! I just built a new temporary npm package based on 1ba732c. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250220200213.tgz

rokucommunity-bot · 2025-02-22T23:45:20Z

Hey there! I just built a new temporary npm package based on 18dd8db. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250222234304.tgz

rokucommunity-bot · 2025-02-24T13:33:52Z

Hey there! I just built a new temporary npm package based on 6b46ca8. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250224133136.tgz

rokucommunity-bot · 2025-02-24T18:02:28Z

Hey there! I just built a new temporary npm package based on f5af66a. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250224180010.tgz

rokucommunity-bot · 2025-02-24T19:47:06Z

Hey there! I just built a new temporary npm package based on 87502c2. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250224194448.tgz

rokucommunity-bot · 2025-02-24T20:33:20Z

Hey there! I just built a new temporary npm package based on 9f29569. You can download it here or install it by running the following command:

npm install https://github.com/rokucommunity/brighterscript/releases/download/v0.0.0-packages/brighterscript-1.0.0-alpha.44-scope-validation-performance.20250224203107.tgz

markwpearce added 2 commits February 13, 2025 10:59

Changed to using lookup tables for clearing diagnostics based on scop…

e594ac1

…e,tag,segment,etc, instead of looping all diagnostics

Changed scope validation metrics reporting

f77c753

markwpearce added the create-package create a temporary npm package on every commit label Feb 13, 2025

markwpearce added this to the v1.0.0 milestone Feb 13, 2025

Adds fine grained timing metrics for files during scope validation

55e91ba

Do not use file maps on initial validation

2be68e9

TwitchBronBron reviewed Feb 14, 2025

View reviewed changes

Comment thread src/util.ts Outdated

Clean up NamespaceContainer Maps

dd4c33b

TwitchBronBron reviewed Feb 20, 2025

View reviewed changes

Comment thread src/CrossScopeValidator.ts Outdated

markwpearce added 2 commits February 20, 2025 14:03

Using Sets for diagnostic filtering operation

e63e354

Fixed lint error.. using scope.name as key for CrossScopeValidator.pr…

be504ae

…ovidedTreeMap

Removed debug logging

c072b43

markwpearce commented Feb 20, 2025

View reviewed changes

Comment thread src/files/BrsFile.ts

markwpearce commented Feb 20, 2025

View reviewed changes

markwpearce added 2 commits February 20, 2025 15:58

Caching getFile path lookups

cfcd33f

fixed lint error

1ba732c

markwpearce added 2 commits February 21, 2025 10:01

removed Cache.get override

3c0df31

Further refinement to DiagnosticManager filtering

18dd8db

Added more tests for DiagnosticManager

6b46ca8

Do not do scope validation on files that will eventuially be ignored

f5af66a

markwpearce added 2 commits February 24, 2025 15:44

Added a type creation counter

942390a

Added a type creation counter

87502c2

commented out type created logs

9f29569

TwitchBronBron approved these changes Feb 24, 2025

View reviewed changes

TwitchBronBron merged commit 37997a8 into release-1.0.0 Feb 24, 2025

TwitchBronBron deleted the scope_validation_performance branch February 24, 2025 20:52

Conversation

markwpearce commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rokucommunity-bot Bot commented Feb 13, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 14, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 14, 2025

Uh oh!

Uh oh!

rokucommunity-bot Bot commented Feb 20, 2025

Uh oh!

Uh oh!

rokucommunity-bot Bot commented Feb 20, 2025

Uh oh!

Uh oh!

markwpearce Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

rokucommunity-bot Bot commented Feb 20, 2025

Uh oh!

markwpearce Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

markwpearce commented Feb 20, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 20, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 22, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 24, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 24, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 24, 2025

Uh oh!

rokucommunity-bot Bot commented Feb 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

markwpearce commented Feb 13, 2025 •

edited

Loading