refactor(ast): make AST structs `repr(C)`. #4614

rzvxa · 2024-08-02T14:45:23Z

Make structs #[repr(C)] without reordering them to benchmark the performance implications of it.

This PR adds 136 bytes of padding in total. You can visit the list of these types here(checkout the assert_layouts.rs diff).

Update:

Doesn't seem too bad!

Linter: We can easily get more than 1% performance gain in the linter to even things out
Prepass: If we fix our issue with Rust 1.80.0 we gain 13% And it should also be possible to find some other areas there to gain back this one percent.

The most notable thing is that parser isn't impacted by this, That is our most optimized crate which every percent counts(and it can be hard to gain back any perf regression there).

graphite-app · 2024-08-02T14:45:28Z

Your org has enabled the Graphite merge queue for merging into main

Add the label “merge” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

rzvxa · 2024-08-02T14:45:37Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rzvxa and the rest of your teammates on Graphite

codspeed-hq · 2024-08-02T14:52:54Z

CodSpeed Performance Report

Merging #4614 will not alter performance

_{Comparing 08-02-refactor_ast_make_ast_structs_repr_c_ (3f53b6f) with main (bcfa297)}

Summary

✅ 32 untouched benchmarks

overlookmotel · 2024-08-03T07:52:20Z

I've made lengthy commment discussing our options on #4296 (comment).

Personally, I'm in favour of trying to avoid as much of this perf hit as we can, even though it's small. And I don't think it'd be too hard/intrusive to do so.

overlookmotel · 2024-08-03T12:14:31Z

@rzvxa and I have discussed. Our plan is to accept the small perf loss for now, because we feel that we'll be able to win it back with the workaround for Rust Analyser described in #4296 (comment). So, in the spirit of Oxc's action policy, merging this now.

graphite-app · 2024-08-03T12:18:46Z

Merge activity

Aug 3, 8:18 AM EDT: The merge label 'merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
Aug 3, 8:23 AM EDT: overlookmotel added this pull request to the Graphite merge queue.
Aug 3, 8:26 AM EDT: overlookmotel merged this pull request with the Graphite merge queue.

Make structs `#[repr(C)]` without reordering them to benchmark the performance implications of it. This PR adds 136 bytes of padding in total. You can visit the list of these types [here](https://github.com/oxc-project/oxc/pull/4404/files/)(checkout the `assert_layouts.rs` diff). Update: Doesn't seem too bad! ![image](https://github.com/user-attachments/assets/05ac9cc8-7cb0-4ed5-9220-44f65c7f5bb9) * Linter: We can easily get more than 1% performance gain in the linter to even things out * Prepass: If we fix our issue with Rust `1.80.0` we gain 13% And it should also be possible to find some other areas there to gain back this one percent. The most notable thing is that `parser` isn't impacted by this, That is our most optimized crate which every percent counts(and it can be hard to gain back any perf regression there).

## [0.23.1] - 2024-08-06 ### Features - fd2d9da ast: Improve `AstKind::debug_name` (#4553) (DonIsaac) - b3b7028 ast: Implement missing Clone, Hash, and Display traits for literals (#4552) (DonIsaac) - 54047e0 ast: `GetSpanMut` trait (#4609) (overlookmotel) - eae401c ast, ast_macros: Apply stable repr to all `#[ast]` enums (#4373) (rzvxa) - 0c52c0d ast_codegen: Add alignment and size data to the schema. (#4615) (rzvxa) - 229a0e9 minifier: Implement dot define for member expressions (#3959) (camc314) - e42ac3a sourcemap: Add `ConcatSourceMapBuilder::from_sourcemaps` (#4639) (overlookmotel) ### Bug Fixes - 4a56954 codegen: Print raw if value is number is Infinity (#4676) (Boshen) - bf48c7f minifier: Fix `keep_var` keeping vars from arrow functions (#4680) (Boshen) - 9be29af minifier: Temporarily fix shadowed `undefined` variable (#4678) (Boshen) - e8b662a minifier: Various fixes to pass minifier conformance (#4667) (Boshen) - a40a217 parser: Parse `assert` keyword in `TSImportAttributes` (#4610) (Boshen) - 03c643a semantic: Incorrect `scope_id` for catch parameter symbols (#4659) (Dunqing) - 6c612d1 semantic/jsdoc: Handle whitespace absence (#4642) (leaysgur) - 0d2c41a semantic/jsdoc: Panic on parsing `type_name_comment`. (#4632) (rzvxa) - 9f8f299 syntax: Prevent creating invalid u32 IDs (#4675) (overlookmotel) - 5327acd transformer/react: The `require` IdentifierReference does not have a `reference_id` (#4658) (Dunqing) - 3987665 transformer/typescript: Incorrect enum-related `symbol_id`/`reference_id` (#4660) (Dunqing) - 4efd54b transformer/typescript: Incorrect `SymbolFlags` for jsx imports (#4549) (Dunqing) ### Performance - 8dd76e4 codegen: Reduce size of `LineOffsetTable` (#4643) (overlookmotel) - b8e6753 codegen: `u32` indexes in `LineOffsetTable` for source maps (#4641) (overlookmotel) - 6ff200d linter: Change react rules and utils to use `Cow` and `CompactStr` instead of `String` (#4603) (DonIsaac) - 0f5e982 minifier: Only visit arrow expression after dropping `console.log` (#4677) (Boshen) - ff43dff sourcemap: Speed up VLQ encoding (#4633) (overlookmotel) - a330773 sourcemap: Reduce string copying in `ConcatSourceMapBuilder` (#4638) (overlookmotel) - 372316b sourcemap: `ConcatSourceMapBuilder` extend `source_contents` in separate loop (#4634) (overlookmotel) - c7f1d48 sourcemap: Keep local copy of previous token in VLQ encode (#4596) (overlookmotel) - 590d795 sourcemap: Shorten main loop encoding VLQ (#4586) (overlookmotel) ### Documentation - c69ada4 ast: Improve AST node documentation (#4051) (Rintaro Itokawa) ### Refactor - ba70001 ast: Put `assert_layouts.rs` behind `debug_assertions` (#4621) (rzvxa) - 3f53b6f ast: Make AST structs `repr(C)`. (#4614) (rzvxa) - 452e0ee ast: Remove defunct `visit_as` + `visit_args` attrs from `#[ast]` macro (#4599) (overlookmotel) - e78cba6 minifier: Ast passes infrastructure (#4625) (Boshen) - d25dea7 parser: Use `ast_builder` in more places. (#4612) (rzvxa) - 09d9822 semantic: Simplify setting scope flags (#4674) (overlookmotel) - 6e453db semantic: Simplify inherit scope flags from parent scope (#4664) (Dunqing)- 9b51e04 Overhaul napi transformer package (#4592) (DonIsaac) ### Testing - 49d5196 ast: Fix `assert_layouts.rs` offset tests on 32bit platforms. (#4620) (rzvxa) Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>

## [0.24.0] - 2024-08-08 - 75f2207 traverse: [**BREAKING**] Replace `find_scope` with `ancestor_scopes` returning iterator (#4693) (overlookmotel) - 506709f traverse: [**BREAKING**] Replace `find_ancestor` with `ancestors` returning iterator (#4692) (overlookmotel) ### Features - 23b0040 allocator: Introduce `CloneIn` trait. (#4726) (rzvxa) - 51c1ca0 ast: Derive `CloneIn` for AST types, using `generate_derive`. (#4732) (rzvxa) - e12bd1e ast: Allow conversion from TSAccessibility into &'static str (#4711) (DonIsaac) - fd2d9da ast: Improve `AstKind::debug_name` (#4553) (DonIsaac) - b3b7028 ast: Implement missing Clone, Hash, and Display traits for literals (#4552) (DonIsaac) - 54047e0 ast: `GetSpanMut` trait (#4609) (overlookmotel) - eae401c ast, ast_macros: Apply stable repr to all `#[ast]` enums (#4373) (rzvxa) - ec0b4cb ast_codegen: Add `derive_clone_in` generator. (#4731) (rzvxa) - 2e91ad6 ast_codegen: Support for `generate_derive` marker. (#4728) (rzvxa) - 82e2f6b ast_codegen: Process AST-related `syntax` types. (#4694) (rzvxa) - 0c52c0d ast_codegen: Add alignment and size data to the schema. (#4615) (rzvxa) - 07607d3 ast_codegen, span: Process `Span` through ast_codegen (#4703) (overlookmotel) - 125c5fd ast_codegen, span: Process `SourceType` through ast_codegen. (#4696) (rzvxa) - eaddc8f linter: Add fixer for eslint/func_names (#4714) (DonIsaac) - 229a0e9 minifier: Implement dot define for member expressions (#3959) (camc314) - 33f1312 semantic: Impl GetSpan for AstNode (#4717) (DonIsaac) - e42ac3a sourcemap: Add `ConcatSourceMapBuilder::from_sourcemaps` (#4639) (overlookmotel) - 2e63618 span: Implement `CloneIn` for the AST-related items. (#4729) (rzvxa) - 6a36616 syntax: Derive `CloneIn` for the AST-related items. (#4730) (rzvxa) ### Bug Fixes - 4a56954 codegen: Print raw if value is number is Infinity (#4676) (Boshen) - 94d3c31 minifier: Avoid removing function declaration from `KeepVar` (#4722) (Boshen) - bf43148 minifier: Do not `remove_syntax` in dead_code_elimination (Boshen) - bf48c7f minifier: Fix `keep_var` keeping vars from arrow functions (#4680) (Boshen) - 9be29af minifier: Temporarily fix shadowed `undefined` variable (#4678) (Boshen) - e8b662a minifier: Various fixes to pass minifier conformance (#4667) (Boshen) - 01d85de napi/transform: Update napi files (Boshen) - f290191 oxc_ast_macros: Fix `syn` lacking features to build (Boshen) - a40a217 parser: Parse `assert` keyword in `TSImportAttributes` (#4610) (Boshen) - 03c643a semantic: Incorrect `scope_id` for catch parameter symbols (#4659) (Dunqing) - 6c612d1 semantic/jsdoc: Handle whitespace absence (#4642) (leaysgur) - 0d2c41a semantic/jsdoc: Panic on parsing `type_name_comment`. (#4632) (rzvxa) - 9f8f299 syntax: Prevent creating invalid u32 IDs (#4675) (overlookmotel) - 4797eaa transformer: Strip TS statements from for in/of statement bodies (#4686) (overlookmotel) - 5327acd transformer/react: The `require` IdentifierReference does not have a `reference_id` (#4658) (Dunqing) - 3987665 transformer/typescript: Incorrect enum-related `symbol_id`/`reference_id` (#4660) (Dunqing) - 4efd54b transformer/typescript: Incorrect `SymbolFlags` for jsx imports (#4549) (Dunqing) ### Performance - 8dd76e4 codegen: Reduce size of `LineOffsetTable` (#4643) (overlookmotel) - b8e6753 codegen: `u32` indexes in `LineOffsetTable` for source maps (#4641) (overlookmotel) - 6ff200d linter: Change react rules and utils to use `Cow` and `CompactStr` instead of `String` (#4603) (DonIsaac) - 0f5e982 minifier: Only visit arrow expression after dropping `console.log` (#4677) (Boshen) - ff43dff sourcemap: Speed up VLQ encoding (#4633) (overlookmotel) - a330773 sourcemap: Reduce string copying in `ConcatSourceMapBuilder` (#4638) (overlookmotel) - 372316b sourcemap: `ConcatSourceMapBuilder` extend `source_contents` in separate loop (#4634) (overlookmotel) - c7f1d48 sourcemap: Keep local copy of previous token in VLQ encode (#4596) (overlookmotel) - 590d795 sourcemap: Shorten main loop encoding VLQ (#4586) (overlookmotel) ### Documentation - c69ada4 ast: Improve AST node documentation (#4051) (Rintaro Itokawa) ### Refactor - 579b797 ast: Use type identifier instead of `CloneIn::Cloned` GAT. (#4738) (rzvxa) - 475266d ast: Use correct lifetimes for name-related methods (#4712) (DonIsaac) - 83b6ca9 ast: Add explicit enum discriminants. (#4689) (rzvxa) - ba70001 ast: Put `assert_layouts.rs` behind `debug_assertions` (#4621) (rzvxa) - 3f53b6f ast: Make AST structs `repr(C)`. (#4614) (rzvxa) - 452e0ee ast: Remove defunct `visit_as` + `visit_args` attrs from `#[ast]` macro (#4599) (overlookmotel) - 2218340 ast, ast_codegen: Use `generate_derive` for implementing `GetSpan` and `GetSpanMut` traits. (#4735) (rzvxa) - fbfd852 minifier: Add `NodeUtil` trait for accessing symbols on ast nodes (#4734) (Boshen) - e0832f8 minifier: Use `oxc_traverse` for AST passes (#4725) (Boshen) - 17602db minifier: Move tests and files around (Boshen) - 3289477 minifier: Clean up tests (#4724) (Boshen) - e78cba6 minifier: Ast passes infrastructure (#4625) (Boshen) - d25dea7 parser: Use `ast_builder` in more places. (#4612) (rzvxa) - 09d9822 semantic: Simplify setting scope flags (#4674) (overlookmotel) - 6e453db semantic: Simplify inherit scope flags from parent scope (#4664) (Dunqing) - e1429e5 span: Reduce #[cfg_attr] boilerplate in type defs (#4702) (overlookmotel) - e24fb5b syntax: Add explicit enum discriminants to AST related types. (#4691) (rzvxa) - 3f3cb62 syntax, span: Reduce #[cfg_attr] boilerplate in type defs (#4698) (overlookmotel) - 54f9897 traverse: Simpler code for entering/exiting unconditional scopes (#4685) (overlookmotel) - 83546d3 traverse: Enter node before entering scope (#4684) (overlookmotel)- 9b51e04 Overhaul napi transformer package (#4592) (DonIsaac) ### Testing - 49d5196 ast: Fix `assert_layouts.rs` offset tests on 32bit platforms. (#4620) (rzvxa) Co-authored-by: Boshen <1430279+Boshen@users.noreply.github.com>

This was referenced Aug 2, 2024

perf(ast, ast_codegen): optimize AST structs' memory layouts #4404

Closed

refactor(ast_codegen): more versatile generation pipeline. #4540

Closed

rzvxa mentioned this pull request Aug 2, 2024

refactor(parser): use ast_builder in more places. #4612

Merged

rzvxa changed the title ~~refactor(ast): make AST structs repr C.~~ refactor(ast): make AST structs repr(C). Aug 2, 2024

rzvxa added the A-ast Area - AST label Aug 2, 2024

rzvxa requested a review from overlookmotel August 2, 2024 14:57

rzvxa marked this pull request as ready for review August 2, 2024 14:59

rzvxa mentioned this pull request Aug 2, 2024

feat(ast_codegen): add alignment and size data to the schema. #4615

Merged

rzvxa force-pushed the 08-02-refactor_ast_make_ast_structs_repr_c_ branch from 53f49fa to 8d21fdb Compare August 2, 2024 17:25

rzvxa changed the base branch from 08-02-refactor_parser_use_ast_builder_in_more_places to 08-02-fix_ast_codegen_detect_complex_type_wrappers August 2, 2024 17:25

rzvxa mentioned this pull request Aug 2, 2024

fix(ast_codegen): detect "complex" type wrappers #4617

Merged

Boshen force-pushed the 08-02-fix_ast_codegen_detect_complex_type_wrappers branch from 926ef49 to 67c5eb2 Compare August 2, 2024 22:42

Boshen force-pushed the 08-02-refactor_ast_make_ast_structs_repr_c_ branch from 8d21fdb to 27e5f76 Compare August 2, 2024 22:42

overlookmotel mentioned this pull request Aug 3, 2024

Make AST #[repr(C)] #4296

Closed

overlookmotel force-pushed the 08-02-fix_ast_codegen_detect_complex_type_wrappers branch from 67c5eb2 to bcfa297 Compare August 3, 2024 12:11

overlookmotel changed the base branch from 08-02-fix_ast_codegen_detect_complex_type_wrappers to main August 3, 2024 12:14

overlookmotel force-pushed the 08-02-refactor_ast_make_ast_structs_repr_c_ branch from 27e5f76 to 941535c Compare August 3, 2024 12:14

overlookmotel added the 0-merge Merge with Graphite Merge Queue label Aug 3, 2024

overlookmotel force-pushed the 08-02-refactor_ast_make_ast_structs_repr_c_ branch from 0fdd209 to 3f53b6f Compare August 3, 2024 12:23

graphite-app bot merged commit 3f53b6f into main Aug 3, 2024
24 checks passed

graphite-app bot deleted the 08-02-refactor_ast_make_ast_structs_repr_c_ branch August 3, 2024 12:26

This was referenced Aug 3, 2024

Add explicit discriminants to all enums in #[ast] macro oxc-project/backlog#123

Open

Apply #[ast] macro to AST types in oxc_syntax crate #4623

Closed

oxc-bot mentioned this pull request Aug 6, 2024

Release crates v0.23.1 #4681

Merged

oxc-bot mentioned this pull request Aug 8, 2024

Release crates v0.24.0 #4749

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(ast): make AST structs `repr(C)`. #4614

refactor(ast): make AST structs `repr(C)`. #4614

rzvxa commented Aug 2, 2024 •

edited

Loading

graphite-app bot commented Aug 2, 2024

rzvxa commented Aug 2, 2024 •

edited

Loading

codspeed-hq bot commented Aug 2, 2024 •

edited

Loading

overlookmotel commented Aug 3, 2024

overlookmotel commented Aug 3, 2024

graphite-app bot commented Aug 3, 2024 •

edited

Loading

refactor(ast): make AST structs repr(C). #4614

refactor(ast): make AST structs repr(C). #4614

Conversation

rzvxa commented Aug 2, 2024 • edited Loading

graphite-app bot commented Aug 2, 2024

Your org has enabled the Graphite merge queue for merging into main

rzvxa commented Aug 2, 2024 • edited Loading

codspeed-hq bot commented Aug 2, 2024 • edited Loading

CodSpeed Performance Report

Merging #4614 will not alter performance

Summary

overlookmotel commented Aug 3, 2024

overlookmotel commented Aug 3, 2024

graphite-app bot commented Aug 3, 2024 • edited Loading

Merge activity

refactor(ast): make AST structs `repr(C)`. #4614

refactor(ast): make AST structs `repr(C)`. #4614

rzvxa commented Aug 2, 2024 •

edited

Loading

rzvxa commented Aug 2, 2024 •

edited

Loading

codspeed-hq bot commented Aug 2, 2024 •

edited

Loading

graphite-app bot commented Aug 3, 2024 •

edited

Loading