refactor(compiler): refactor fory compiler into hierarchical architecture #3179
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why?
The current Fory compiler mixes FDL-native and protobuf-compatible syntax handling in a single parser, making it difficult to add support for new IDL formats like .proto and .fbs files. The validation logic is scattered across parsing and Schema.validate(), and there's no clear separation between parsing, semantic analysis, and code generation.
What does this PR do?
This PR refactors the Fory compiler into a hierarchical, multi-frontend architecture that establishes the Fory IDL AST as the canonical intermediate representation (IR), with separate frontend parsers for different IDL formats.
Key changes:
New directory structure with clear separation of concerns:
ir/- Intermediate Representation (canonical Fory AST)ast.py- Core AST node definitions withSourceLocationtrackingtypes.py- Extended type system (primitives including varint, tagged types, etc.)validator.py- Centralized semantic validationemitter.py- FDL text emitter for debugging translated schemasfrontend/- IDL Frontendsbase.py- Base frontend interfacefdl/- FDL Frontend (lexer + parser)proto/- Protobuf Frontend (lexer + parser + translator to Fory IR)fbs/- FlatBuffers Frontend (placeholder)Proto3 frontend - Full support for parsing
.protofiles and translating to Fory IR:(fory).id,(fory).ref,(fory).nullable, etc.)Simplified FDL syntax - Removed protobuf-style
(fory)prefix from options:option use_record_for_java_message = true;message Foo [id=100] { ... }MyType data = 1 [ref=true, nullable=true];Extended type system with new primitive kinds:
int8-int64,uint8-uint64varint32,varint64,var_uint32,var_uint64tagged_int64,tagged_uint64float16,duration,decimalImproved code generators for all target languages with better type mapping
CLI enhancements:
.fdl,.proto)--emit-fdlflag to output translated FDL for debuggingCross-language integration tests for proto-based schemas
Related issues
Closes #3178
Does this PR introduce any user-facing change?
CLI now accepts
.protofiles directly (in addition to.fdl)FDL syntax simplified:
option (fory).xxx→option xxxNew primitive types available in FDL
Does this PR introduce any public API change?
Does this PR introduce any binary protocol compatibility change?
Benchmark
N/A - This is a compiler refactoring that doesn't affect runtime performance.