style: Add macro preprocessing to fix tree-sitter parse errors#2559
style: Add macro preprocessing to fix tree-sitter parse errors#2559bobtista wants to merge 5 commits intoTheSuperHackers:mainfrom
Conversation
|
| Filename | Overview |
|---|---|
| scripts/cpp/convert_leading_spaces_to_tabs.py | New tree-sitter based tool that preprocesses C++ macros before parsing to fix indentation; one dead clause in block-comment detection and minor style nit |
| Core/GameEngine/Include/Common/GameAudio.h | Whitespace-only: mixed-spaces indentation on several virtual method declarations corrected to tabs |
| Core/GameEngine/Include/Common/FileSystem.h | Whitespace-only: two private member declarations corrected from space-indented to tab-indented |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Read file as cp1252] --> B[Encode to UTF-8 bytes]
B --> C[preprocess_for_parsing\nexpand CALLBACK/GCALL/WINAPI/IN/OUT/etc.\nwith same-length space replacements]
C --> D[tree-sitter parse\npreprocessed bytes]
D --> E{Top-level ERROR\nnodes > 10?}
E -- Yes --> F[Skip file entirely]
E -- No --> G[For each line with leading spaces]
G --> H{In block comment\nor macro continuation?}
H -- Yes --> I[Preserve line as-is]
H -- No --> J[Find AST node at\nfirst non-whitespace col]
J --> K{ERROR ancestor\nor continuation line?}
K -- Yes --> I
K -- No --> L[get_indent_depth\ncount scope-creating ancestors]
L --> M[Apply brace / case /\naccess-specifier overrides]
M --> N{depth == 0 but\n>= 4 leading spaces?}
N -- Yes --> I
N -- No --> O[Replace leading WS\nwith depth x TAB]
O --> P[Write original file\nwith new indentation]
Prompt To Fix All With AI
This is a comment left during a code review.
Path: scripts/cpp/convert_leading_spaces_to_tabs.py
Line: 177
Comment:
**Dead second clause in block-comment guard**
When `stripped.startswith('/*')`, `stripped.index('/*')` is always `0`, so `stripped.index('/*') + 2` is always `2`. A `*/` closing token can appear at index 2 at earliest (i.e. `/**/`), making `stripped.index('*/') < 2` always `False`. The entire second clause of the `or` is therefore unreachable and the condition collapses to just `'*/' not in stripped`. Consider simplifying:
```suggestion
if '*/' not in stripped:
in_block_comment = True
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: scripts/cpp/convert_leading_spaces_to_tabs.py
Line: 123-127
Comment:
**Excess blank lines between functions**
PEP 8 recommends two blank lines between top-level definitions. There are four blank lines here between `is_continuation_line` and `process_file`.
```suggestion
```
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "style: Convert additional lines to tabs ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| continue | ||
|
|
||
| if stripped.startswith('/*'): | ||
| if '*/' not in stripped or stripped.index('*/') < stripped.index('/*') + 2: |
There was a problem hiding this comment.
When stripped.startswith('/*'), stripped.index('/*') is always 0, so stripped.index('/*') + 2 is always 2. A */ closing token can appear at index 2 at earliest (i.e. /**/), making stripped.index('*/') < 2 always False. The entire second clause of the or is therefore unreachable and the condition collapses to just '*/' not in stripped. Consider simplifying:
| if '*/' not in stripped or stripped.index('*/') < stripped.index('/*') + 2: | |
| if '*/' not in stripped: | |
| in_block_comment = True |
Prompt To Fix With AI
This is a comment left during a code review.
Path: scripts/cpp/convert_leading_spaces_to_tabs.py
Line: 177
Comment:
**Dead second clause in block-comment guard**
When `stripped.startswith('/*')`, `stripped.index('/*')` is always `0`, so `stripped.index('/*') + 2` is always `2`. A `*/` closing token can appear at index 2 at earliest (i.e. `/**/`), making `stripped.index('*/') < 2` always `False`. The entire second clause of the `or` is therefore unreachable and the condition collapses to just `'*/' not in stripped`. Consider simplifying:
```suggestion
if '*/' not in stripped:
in_block_comment = True
```
How can I resolve this? If you propose a fix, please make it concise.
1 participant
Summary
Builds on #2554 (Core mixed-whitespace PR). All C++ changes are whitespace-only.
What this does
Adds a preprocessing step that expands known macros (GCALL, CALLBACK, WINAPI,
IN, OUT, RO, NEW, W3DNEW, __RPC_FAR, __asm, etc.) before passing source to
tree-sitter for parsing. This fixes parse errors that caused the tool to skip
lines it couldn't confidently reindent.
The expansions are same-length replacements to preserve byte offsets, and are
only used for parsing — the original source content is preserved in output.
After #2554 merges to main, this PR should be rebased onto main so the diff
only shows the Phase 2 delta (5 files changed).