Skip to content

Incorrect tokenization of multiline comments with whitespace #506

@alexr00

Description

@alexr00

Checklist

  • This problem exists even with the setting "C_Cpp.enhancedColorization": "Disabled"
    This bug exists for C
    This bug exists for C++
    This bug exists for Objective-C
    This bug exists for Objective-C++

The code with a problem is:

#define TCP_NOTSENT_LOWAT	 25 /* Limit number of unsent bytes in

				       write queue.  */

#define TCP_CC_INFO		 26 /* Get Congestion Control

				       (optional) info.  */

#define TCP_SAVE_SYN		 27 /* Record SYN headers for new

				       connections.  */

#define TCP_SAVED_SYN		 28 /* Get SYN headers recorded for

				       connection.  */

It looks like:

** screenshot + theme name preferable **
image

It should look like:

Comments should span multiple lines.

Originally from @duzeev in microsoft/vscode#107565

Activity

jeff-hykin

jeff-hykin commented on Nov 4, 2020

@jeff-hykin
Owner

I think this is unfixable without the tree sitter.

The #define uses the while pattern to ignore ranges (ranges like a { with no }), but in doing so, it ignores the range of a comment too.

matter123

matter123 commented on Nov 5, 2020

@matter123
Collaborator

@jeff-hykin Apologies for not being involved for the last 6 months.

The reason this breaks is that :single_line_macro matches as the line does not end in \

It should be possible to have :single_line_macro exclude lines that have /* that is not later succeeded by */. This would cause the macro matching to fall back to multi-line macro matching where the comment would be properly matched.

jeff-hykin

jeff-hykin commented on Nov 6, 2020

@jeff-hykin
Owner

You're good @matter123 , I've been gone for awhile, and I probably will be intermittent at best. I've gone back to just using master since it's too much effort with the branches when I have such short time.

Thanks for the comment, I forgot there was a single line macro. Still sounds like a hack to me so I'll let you implement that one 😆. If there was a string with /* inside of it, seems like that would be a problem.

added a commit that references this issue on Nov 6, 2020
9c4d17e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Nearly Impossible :(Things that likely need more than just Textmate matching to fix

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Participants

      @matter123@jeff-hykin@alexr00

      Issue actions

        Incorrect tokenization of multiline comments with whitespace · Issue #506 · jeff-hykin/better-cpp-syntax