Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Token hints for missing closing braces: classes, enums, jsx, modules, types #37497

Closed
wants to merge 1 commit into from

Conversation

ayazhafiz
Copy link
Contributor

This commit adds token hints for missing close braces in

  • Class definitions
  • Enum definitions
  • JSX expressions
  • Module definitions (this includes augmentations and namespaces)
  • Type member lists (this includes interfaces)

The token hint refers to the opening brace for which the parser expected
a corresponding closing brace.

Note that this commit does not update all occurences of an expectation
for a close brace to provide token hints. Constructs for which this
token hint is not provided include:

  • Mapped types
  • Switch statements
  • Object binding patterns
  • JSX spread attributes (<div {...props}></div>)
  • JSDoc implements/augments tags (/* @implements {SomeInterface} */)

Part of #35597.

@ayazhafiz
Copy link
Contributor Author

cc @sheonhan

@ayazhafiz
Copy link
Contributor Author

ayazhafiz commented Mar 20, 2020

Shoot, I will fix the whitespace changes. Sorry about that.

edit: done

@sandersn sandersn added this to Not started in PR Backlog Apr 1, 2020
@sandersn sandersn added the For Backlog Bug PRs that fix a backlog bug label Apr 1, 2020
@sandersn sandersn moved this from Not started to Needs review in PR Backlog Apr 1, 2020
@ayazhafiz
Copy link
Contributor Author

@andrewbranch @DanielRosenwasser is there any additional action I could take to get a review on this?

@DanielRosenwasser
Copy link
Member

@typescript-bot test this
@typescript-bot user test this
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 14, 2020

Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at f2d57da. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 14, 2020

Heya @DanielRosenwasser, I've started to run the parallelized community code test suite on this PR at f2d57da. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 14, 2020

Heya @DanielRosenwasser, I've started to run the extended test suite on this PR at f2d57da. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

The user suite test run you requested has finished and failed. I've opened a PR with the baseline diff from master.

… types

This commit adds token hints for missing close braces in

- Class definitions
- Enum definitions
- JSX expressions
- Module definitions (this includes augmentations and namespaces)
- Type member lists (this includes interfaces)

The token hint refers to the opening brace for which the parser expected
a corresponding closing brace.

Note that this commit *does not* update all occurences of an expectation
for a close brace to provide token hints. Constructs for which this
token hint is not provided include:

- Mapped types
- Switch statements
- Object binding patterns
- JSX spread attributes (`<div {...props}></div>`)
- JSDoc implements/augments tags (`/* @implements {SomeInterface} */`)

Part of microsoft#35597.
PR Backlog automation moved this from Needs review to Needs merge May 14, 2020
andrewbranch
andrewbranch previously approved these changes May 14, 2020
Copy link
Member

@andrewbranch andrewbranch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DanielRosenwasser I guess the extended tests didn’t have any unbalanced braces. Code and baselines look good to me. Anyone else want to review first?

@DanielRosenwasser
Copy link
Member

I guess the extended tests didn’t have any unbalanced braces.

I figured, just wanted to make sure that we weren't introducing any weird behavioral regressions

@DanielRosenwasser
Copy link
Member

DanielRosenwasser commented May 15, 2020

@weswigham perf tests seem to have had an issue. Any clue?

/usr/bin/npm uninstall @octokit/rest --no-save
Unhandled rejection Error: EACCES: permission denied, open '/home/setup/.npm/_cacache/index-v5/4d/74/9d5c4ed5545c7788de2803ecc3b7d9041eb8de651d84decfd91be1a07c77'

https://typescript.visualstudio.com/TypeScript/_build/results?buildId=73871&view=logs&j=db40ead0-d549-5fb8-3f02-9e9562a5d19f&t=6c3d3f20-8d71-550b-6f99-a92788a798fa&l=30

Do we just need to pass --force in a config file somewhere? 😬

@weswigham
Copy link
Member

weswigham commented May 15, 2020

I pinged @rbuckton in one of my PRs about the same thing; I assumed it's because the version of npm on the perf machine is out of date, but it could be anything. The perf machine is managed by us, and by "managed by", I mean, "is a box still sitting in our office in redmond", so it could need some regular maintenence.

Copy link
Member

@DanielRosenwasser DanielRosenwasser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it should've been obvious, the previous approach is too hacky and mucks with non-local state that it can't be sure it created.

Consider making parseErrorAtPosition return the diagnostic it creates if it can. Then wire up parseExpectedCloseToken to use that, directly or indirectly so that it doesn't have to reach into the diagnostics list.

@@ -14,6 +14,7 @@ tests/cases/compiler/classMemberWithMissingIdentifier2.ts(3,1): error TS1128: De
!!! error TS1146: Declaration expected.
~
!!! error TS1005: ';' expected.
!!! related TS1007 tests/cases/compiler/classMemberWithMissingIdentifier2.ts:1:9: The parser expected to find a '}' to match the '{' token here.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it looks like the heuristic on this is actually...not working correctly given

// Don't report another error if it would just be at the same position as the last error.

PR Backlog automation moved this from Needs merge to Waiting on author May 15, 2020
@andrewbranch
Copy link
Member

@DanielRosenwasser I initially had the same thought about looking at parseDiagnostics, but I took a look and there are lots of other parsing functions that use that tactic. And when parseExpected is called without a diagnostic message and it returns false, it necessarily added Diagnostics._0_expected to the array. If it’s simple to clean that up a bit I’m all for it, but I’m not sure peeking at parseDiagnostics is actually the culprit here 🤔

@DanielRosenwasser
Copy link
Member

I initially had the same thought about looking at parseDiagnostics, but I took a look and there are lots of other parsing functions that use that tactic

Then I think those functions are also bad 🙂

@rbuckton
Copy link
Member

The perf test issue should be resolved, I think.

@rbuckton
Copy link
Member

@typescript-bot perf test

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 15, 2020

Heya @rbuckton, I've started to run the perf test suite on this PR at 35cd0e4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@rbuckton
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..37497

Metric master 37497 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 328,171k (± 0.03%) 327,664k (± 0.03%) -507k (- 0.15%) 327,330k 327,798k
Parse Time 1.65s (± 0.65%) 1.65s (± 0.62%) -0.00s (- 0.12%) 1.63s 1.68s
Bind Time 0.91s (± 0.95%) 0.90s (± 0.99%) -0.00s (- 0.44%) 0.88s 0.92s
Check Time 4.81s (± 0.77%) 4.85s (± 0.52%) +0.04s (+ 0.81%) 4.78s 4.91s
Emit Time 5.44s (± 1.17%) 5.41s (± 1.23%) -0.03s (- 0.61%) 5.29s 5.60s
Total Time 12.82s (± 0.74%) 12.82s (± 0.51%) -0.00s (- 0.02%) 12.71s 12.99s
Monaco - node (v10.16.3, x64)
Memory used 327,230k (± 0.02%) 327,303k (± 0.02%) +73k (+ 0.02%) 327,152k 327,452k
Parse Time 1.28s (± 0.61%) 1.27s (± 0.70%) -0.00s (- 0.31%) 1.26s 1.29s
Bind Time 0.79s (± 0.28%) 0.78s (± 0.46%) -0.00s (- 0.63%) 0.78s 0.79s
Check Time 4.84s (± 0.44%) 4.88s (± 0.53%) +0.04s (+ 0.72%) 4.83s 4.94s
Emit Time 2.96s (± 0.97%) 2.95s (± 0.49%) -0.01s (- 0.30%) 2.92s 2.99s
Total Time 9.87s (± 0.48%) 9.88s (± 0.29%) +0.02s (+ 0.15%) 9.82s 9.94s
TFS - node (v10.16.3, x64)
Memory used 292,345k (± 0.02%) 292,432k (± 0.03%) +87k (+ 0.03%) 292,220k 292,586k
Parse Time 0.97s (± 0.82%) 0.96s (± 0.79%) -0.01s (- 0.72%) 0.95s 0.99s
Bind Time 0.76s (± 0.59%) 0.75s (± 1.03%) -0.01s (- 0.92%) 0.74s 0.77s
Check Time 4.37s (± 0.49%) 4.38s (± 0.40%) +0.02s (+ 0.37%) 4.35s 4.42s
Emit Time 3.11s (± 0.81%) 3.08s (± 1.15%) -0.03s (- 0.93%) 2.97s 3.14s
Total Time 9.21s (± 0.45%) 9.18s (± 0.45%) -0.02s (- 0.27%) 9.07s 9.27s
material-ui - node (v10.16.3, x64)
Memory used 449,320k (± 0.01%) 449,969k (± 0.01%) +649k (+ 0.14%) 449,825k 450,060k
Parse Time 1.80s (± 0.46%) 1.79s (± 0.59%) -0.01s (- 0.50%) 1.76s 1.81s
Bind Time 0.70s (± 0.68%) 0.70s (± 0.67%) -0.00s (- 0.43%) 0.69s 0.71s
Check Time 12.74s (± 0.79%) 12.83s (± 0.86%) +0.09s (+ 0.70%) 12.68s 13.18s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.24s (± 0.67%) 15.32s (± 0.70%) +0.08s (+ 0.51%) 15.16s 15.66s
Angular - node (v12.1.0, x64)
Memory used 303,683k (± 0.01%) 303,278k (± 0.03%) -405k (- 0.13%) 303,130k 303,582k
Parse Time 1.60s (± 0.44%) 1.60s (± 0.63%) -0.01s (- 0.31%) 1.58s 1.62s
Bind Time 0.89s (± 1.00%) 0.88s (± 0.39%) -0.01s (- 1.23%) 0.88s 0.89s
Check Time 4.72s (± 0.57%) 4.73s (± 0.50%) +0.00s (+ 0.11%) 4.66s 4.78s
Emit Time 5.55s (± 0.74%) 5.52s (± 0.56%) -0.03s (- 0.50%) 5.43s 5.58s
Total Time 12.77s (± 0.35%) 12.73s (± 0.30%) -0.04s (- 0.31%) 12.64s 12.79s
Monaco - node (v12.1.0, x64)
Memory used 307,201k (± 0.02%) 307,212k (± 0.02%) +11k (+ 0.00%) 307,084k 307,372k
Parse Time 1.24s (± 0.55%) 1.23s (± 0.67%) -0.01s (- 0.81%) 1.21s 1.24s
Bind Time 0.76s (± 0.49%) 0.75s (± 1.01%) -0.00s (- 0.00%) 0.74s 0.77s
Check Time 4.67s (± 0.39%) 4.70s (± 0.68%) +0.02s (+ 0.51%) 4.65s 4.79s
Emit Time 3.02s (± 0.72%) 3.00s (± 0.67%) -0.02s (- 0.60%) 2.95s 3.03s
Total Time 9.67s (± 0.37%) 9.67s (± 0.42%) -0.00s (- 0.01%) 9.56s 9.76s
TFS - node (v12.1.0, x64)
Memory used 274,597k (± 0.01%) 274,660k (± 0.02%) +63k (+ 0.02%) 274,492k 274,780k
Parse Time 0.94s (± 0.72%) 0.94s (± 0.51%) -0.01s (- 0.74%) 0.93s 0.95s
Bind Time 0.73s (± 1.21%) 0.72s (± 0.68%) -0.01s (- 1.36%) 0.71s 0.73s
Check Time 4.26s (± 0.72%) 4.29s (± 0.41%) +0.03s (+ 0.66%) 4.26s 4.33s
Emit Time 3.15s (± 0.96%) 3.10s (± 0.63%) -0.05s (- 1.68%) 3.06s 3.15s
Total Time 9.09s (± 0.55%) 9.05s (± 0.25%) -0.04s (- 0.45%) 9.00s 9.11s
material-ui - node (v12.1.0, x64)
Memory used 426,693k (± 0.01%) 427,410k (± 0.01%) +717k (+ 0.17%) 427,310k 427,534k
Parse Time 1.77s (± 0.57%) 1.76s (± 0.49%) -0.02s (- 0.90%) 1.74s 1.78s
Bind Time 0.66s (± 0.84%) 0.65s (± 1.18%) -0.01s (- 1.98%) 0.63s 0.67s
Check Time 11.27s (± 0.89%) 11.42s (± 0.69%) +0.16s (+ 1.41%) 11.27s 11.63s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 13.69s (± 0.79%) 13.83s (± 0.55%) +0.14s (+ 0.99%) 13.70s 14.02s
Angular - node (v8.9.0, x64)
Memory used 323,234k (± 0.02%) 322,732k (± 0.02%) -502k (- 0.16%) 322,607k 322,898k
Parse Time 2.13s (± 0.47%) 2.13s (± 0.44%) -0.00s (- 0.14%) 2.11s 2.15s
Bind Time 0.94s (± 0.72%) 0.94s (± 1.25%) -0.01s (- 0.63%) 0.92s 0.97s
Check Time 5.55s (± 0.45%) 5.52s (± 1.25%) -0.03s (- 0.61%) 5.27s 5.61s
Emit Time 6.34s (± 0.82%) 6.29s (± 1.37%) -0.05s (- 0.80%) 6.13s 6.55s
Total Time 14.97s (± 0.45%) 14.87s (± 0.71%) -0.10s (- 0.67%) 14.71s 15.22s
Monaco - node (v8.9.0, x64)
Memory used 325,847k (± 0.01%) 325,888k (± 0.02%) +41k (+ 0.01%) 325,726k 326,099k
Parse Time 1.56s (± 0.84%) 1.55s (± 0.23%) -0.01s (- 0.32%) 1.55s 1.56s
Bind Time 0.92s (± 1.53%) 0.91s (± 0.91%) -0.01s (- 1.52%) 0.89s 0.93s
Check Time 5.47s (± 0.95%) 5.46s (± 0.37%) -0.01s (- 0.27%) 5.41s 5.50s
Emit Time 3.52s (± 0.80%) 3.49s (± 0.35%) -0.03s (- 0.80%) 3.47s 3.52s
Total Time 11.47s (± 0.55%) 11.41s (± 0.26%) -0.06s (- 0.52%) 11.34s 11.47s
TFS - node (v8.9.0, x64)
Memory used 291,871k (± 0.02%) 291,928k (± 0.01%) +58k (+ 0.02%) 291,824k 292,013k
Parse Time 1.26s (± 0.41%) 1.26s (± 0.26%) -0.00s (- 0.08%) 1.25s 1.27s
Bind Time 0.76s (± 0.68%) 0.75s (± 0.63%) -0.01s (- 0.79%) 0.75s 0.77s
Check Time 5.10s (± 1.35%) 5.10s (± 1.68%) -0.00s (- 0.06%) 4.86s 5.22s
Emit Time 3.18s (± 2.75%) 3.19s (± 2.71%) +0.01s (+ 0.41%) 3.06s 3.43s
Total Time 10.30s (± 0.45%) 10.30s (± 0.31%) +0.01s (+ 0.07%) 10.24s 10.36s
material-ui - node (v8.9.0, x64)
Memory used 452,158k (± 0.01%) 452,897k (± 0.01%) +739k (+ 0.16%) 452,808k 453,004k
Parse Time 2.15s (± 0.97%) 2.12s (± 0.54%) -0.02s (- 1.07%) 2.10s 2.15s
Bind Time 0.83s (± 0.67%) 0.82s (± 1.03%) -0.01s (- 0.85%) 0.81s 0.84s
Check Time 16.91s (± 1.10%) 16.64s (± 0.63%) -0.28s (- 1.64%) 16.47s 16.94s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 19.89s (± 0.96%) 19.58s (± 0.54%) -0.31s (- 1.56%) 19.45s 19.89s
Angular - node (v8.9.0, x86)
Memory used 186,103k (± 0.02%) 185,845k (± 0.03%) -258k (- 0.14%) 185,694k 185,966k
Parse Time 2.07s (± 0.68%) 2.05s (± 0.54%) -0.02s (- 0.96%) 2.04s 2.09s
Bind Time 1.09s (± 0.53%) 1.08s (± 0.48%) -0.01s (- 1.10%) 1.07s 1.09s
Check Time 5.04s (± 0.38%) 5.08s (± 0.74%) +0.04s (+ 0.87%) 5.02s 5.19s
Emit Time 6.09s (± 1.13%) 6.12s (± 1.17%) +0.03s (+ 0.56%) 6.03s 6.38s
Total Time 14.29s (± 0.46%) 14.34s (± 0.61%) +0.05s (+ 0.37%) 14.20s 14.57s
Monaco - node (v8.9.0, x86)
Memory used 185,621k (± 0.01%) 185,665k (± 0.01%) +45k (+ 0.02%) 185,622k 185,752k
Parse Time 1.60s (± 1.27%) 1.60s (± 0.55%) -0.00s (- 0.06%) 1.58s 1.62s
Bind Time 0.78s (± 1.35%) 0.77s (± 0.75%) -0.01s (- 1.40%) 0.76s 0.79s
Check Time 5.50s (± 0.69%) 5.49s (± 0.44%) -0.00s (- 0.07%) 5.44s 5.55s
Emit Time 2.91s (± 0.94%) 2.88s (± 0.92%) -0.03s (- 0.93%) 2.85s 2.98s
Total Time 10.79s (± 0.71%) 10.75s (± 0.32%) -0.04s (- 0.39%) 10.67s 10.81s
TFS - node (v8.9.0, x86)
Memory used 167,161k (± 0.02%) 167,230k (± 0.02%) +69k (+ 0.04%) 167,147k 167,286k
Parse Time 1.30s (± 0.81%) 1.30s (± 0.57%) +0.00s (+ 0.23%) 1.29s 1.32s
Bind Time 0.72s (± 0.51%) 0.71s (± 0.70%) -0.00s (- 0.14%) 0.71s 0.73s
Check Time 4.69s (± 0.53%) 4.70s (± 0.55%) +0.01s (+ 0.32%) 4.65s 4.76s
Emit Time 3.02s (± 2.24%) 3.00s (± 0.45%) -0.03s (- 0.89%) 2.97s 3.04s
Total Time 9.72s (± 0.79%) 9.71s (± 0.36%) -0.01s (- 0.08%) 9.64s 9.79s
material-ui - node (v8.9.0, x86)
Memory used 256,161k (± 0.02%) 256,607k (± 0.01%) +446k (+ 0.17%) 256,535k 256,673k
Parse Time 2.19s (± 0.53%) 2.19s (± 0.86%) -0.01s (- 0.27%) 2.16s 2.25s
Bind Time 0.70s (± 1.43%) 0.69s (± 0.96%) -0.01s (- 1.70%) 0.68s 0.71s
Check Time 15.32s (± 0.83%) 15.48s (± 0.50%) +0.16s (+ 1.05%) 15.36s 15.60s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 18.22s (± 0.67%) 18.36s (± 0.44%) +0.15s (+ 0.81%) 18.22s 18.54s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 37497 10
Baseline master 10

@DanielRosenwasser
Copy link
Member

DanielRosenwasser commented May 15, 2020

Then I think those functions are also bad

Just to be specific, the reason I think so is that

And when parseExpected is called without a diagnostic message and it returns false, it necessarily added Diagnostics._0_expected to the array.

is actually untrue! See the reference over at #37497 (comment). We'll return false, but the diagnostic will be tossed away, even if it's a different diagnostic!

function parseErrorAtPosition(start: number, length: number, message: DiagnosticMessage, arg0?: any): void {
// Don't report another error if it would just be at the same position as the last error.
const lastError = lastOrUndefined(parseDiagnostics);
if (!lastError || start !== lastError.start) {
parseDiagnostics.push(createFileDiagnostic(sourceFile, start, length, message, arg0));
}

I believe the de-duping logic means that you really can never rely on the most recent error in the list, which was always a super sketchy approach anyway.

@andrewbranch
Copy link
Member

Ahh, I wasn’t looking all the way into parseErrorAtPosition, you’re right.

Then I think those functions are also bad

I do too, I just try to avoid refactoring code that’s ostensibly been working fine for a long time. But now it definitely looks like it’s not working fine. My bad. Carry on.

@sandersn sandersn moved this from Waiting on author to Needs review in PR Backlog Feb 16, 2021
@sandersn
Copy link
Member

@DanielRosenwasser @andrewbranch I can't tell what the state of this PR is. Is it the right approach? Does it need to be reworked entirely? If we're waiting for changes from @ayazhafiz, what are the changes? The long comment history made it too hard for me to figure any of this out.

@andrewbranch
Copy link
Member

It needs a refactor in the parser which, on its face, would appear unrelated to the actual goal here, and I’m unsure how big of a refactor that is. Daniel described what needs to happen here, and his review points out an incorrect test baseline that exhibits why the current approach is unreliable.

@sandersn
Copy link
Member

@ayazhafiz Would you like to attempt the refactor? I'll close this PR soon if not.

@ayazhafiz
Copy link
Contributor Author

No, I do not have time to do this presently.

@ayazhafiz ayazhafiz closed this Feb 17, 2021
PR Backlog automation moved this from Needs review to Done Feb 17, 2021
@typescript-bot
Copy link
Collaborator

This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Backlog Bug PRs that fix a backlog bug
Projects
PR Backlog
  
Done
Development

Successfully merging this pull request may close these issues.

None yet

7 participants