refactor(lexer): TS contextual keyword 30개를 identifier로 통합#309
Merged
Conversation
number, string, boolean 등 TS contextual keyword를 개별 토큰(kw_number 등) 대신 일반 identifier로 토큰화하도록 변경. 파서에서 isContextual/eatContextual 헬퍼로 문자열 비교하여 TS 키워드를 판별. 이유: 개별 토큰 방식은 식별자 위치에서 TS 키워드를 누락하는 버그를 유발 (d3의 `export default function number(x)` 등). esbuild/oxc/SWC와 동일하게 identifier로 토큰화하면 이런 류의 버그가 원천 차단됨. 변경: - token.zig: 30개 TS keyword enum 제거, keywords 맵에서 제거 - parser.zig: isContextual/eatContextual/expectContextual/isContextualAny 헬퍼 추가 - ts.zig: parsePrimaryType 등 TS 타입 키워드를 문자열 비교로 변환 - statement.zig: type/namespace/module/declare/abstract를 .identifier arm에서 처리 - expression.zig: as/satisfies를 isContextual로 변환 - module.zig: import/export의 as/type/assert를 헬퍼로 변환 - declaration.zig: class member modifier를 isContextualAny로 변환 - binding.zig: parameter property readonly를 isContextual로 변환 성능: 벤치마크 동일 (10k줄 파싱 ~14ms → ~14ms) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Smoke Test Results (CI)Smoke Test Results
38/38 projects built successfully.
|
Benchmark Results (CI)macos-latesttranspile — small (100 lines)
transpile — medium (1K lines)
transpile — large (5K lines)
bundle — small (10 modules)
bundle — medium (50 modules)
bundle — large (200 modules)
ubuntu-latesttranspile — small (100 lines)
transpile — medium (1K lines)
transpile — large (5K lines)
bundle — small (10 modules)
bundle — medium (50 modules)
bundle — large (200 modules)
windows-latesttranspile — small (100 lines)
transpile — medium (1K lines)
transpile — large (5K lines)
bundle — small (10 modules)
bundle — medium (50 modules)
bundle — large (200 modules)
|
- module.zig: 직접 소스 슬라이싱 → isContextual("source")로 통일
- expression.zig: isContextual 중복 호출 제거 (tokenText 1회 캐시)
- ts.zig: 10개 문자열 비교 체인 → StaticStringMap(ts_type_keywords) 룩업
- declaration.zig: modifier 문자열 목록 → ts_class_modifiers 상수 추출
- parser.zig: expectContextual에서 addError 호출로 통일
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
number,string,boolean등 30개 TS contextual keyword를 개별 토큰(kw_number등) 대신 일반identifier로 토큰화isContextual/eatContextual/expectContextual/isContextualAny헬퍼 추가export default function number(x)같은 버그가 원천 차단됨변경 파일 (8개)
token.zig: 30개 TS keyword enum 제거, keywords 맵에서 제거parser.zig: contextual keyword 헬퍼 4개 추가ts.zig,statement.zig,expression.zig,module.zig,declaration.zig,binding.zig: 토큰 매칭 → 문자열 비교로 전환Test plan
zig build test— 1260개 유닛 테스트 통과scaleLinear(50)= 0.5)🤖 Generated with Claude Code