Diagnostic is shown at incorrect location after multi-line comment

[enricolyb]

Describe the bug
The position of the diagnostic in the VSCode editor is incorrect if a multi-line comment is present in the source file before the referenced position.
This happens when CRLF line endings (Windows) are used.

To Reproduce
Add a multi-line comment using either """ or ''' at the top of the source file with CRLF line endings.
Add incorrect Python code after the multi-line comment.

Expected behavior
The position of the incorrect Python code is marked correctly in VSCode editor.

Screenshots or Code

The green line comes from the built-in Python code checker, location is correct
The red line comes from pyright, location is incorrect

VS Code extension or command-line
VSCode extension version 1.0.16 does not work correctly.
VSCode extension version 1.0.15 works correctly.

Additional context
It seems to be related to the recent changes in tokenizer.ts, line ~973. This check doesn't handle CRLF. The resulting Tokenizer lines collection is not correct.

[erictraut]

Thanks for reporting. Yes, this was a regression I introduced recently. It affects only files formatted with CR/LF. I didn't notice because I tend to use LF only. I've fixed the regression and added a unit test so a similar regression won't occur in the future. The fix will be in the next version (1.0.18).

[erictraut]

This is now fixed in version 1.0.18, which I just published.