Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AST transfer WIP #2457

Draft
wants to merge 34 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
8c0da26
TEMP: CI run on PR head commit
overlookmotel Mar 20, 2024
3151f6c
TEMP: Only run NAPI benchmarks on CI
overlookmotel Mar 21, 2024
11c94a4
TEMP: Cache build files for branch
overlookmotel Mar 21, 2024
ada02a5
AST transfer WIP
overlookmotel Mar 20, 2024
f38c105
Extract strings as string slices
overlookmotel Mar 20, 2024
eebc74e
Simplify pointer calculations
overlookmotel Mar 20, 2024
47c0e83
Copy source into arena
overlookmotel Mar 20, 2024
9757082
Validate UTF8 with `simdutf8`
overlookmotel Mar 20, 2024
3191d13
Limit AST to 2 GiB
overlookmotel Mar 20, 2024
a4e66ce
Use `>>` everywhere
overlookmotel Mar 20, 2024
0b8c632
Reduce bitshift operations
overlookmotel Mar 20, 2024
eeeff64
`generateDeserializerCall` [refactor]
overlookmotel Mar 21, 2024
f3d7b2a
Inline primitive deserializers
overlookmotel Mar 21, 2024
ac1bc2e
Reuse buffer
overlookmotel Mar 21, 2024
13fa060
Use 4 GiB aligned buffer
overlookmotel Mar 21, 2024
49964ee
Remove `Allocator::into_bump`
overlookmotel Mar 22, 2024
6678f10
Panic if `Allocator::set_allocation_limit` called
overlookmotel Mar 22, 2024
156cd68
Rename vars in `Allocator::from_raw_parts`
overlookmotel Mar 22, 2024
999786c
Use 4 GiB size buffer
overlookmotel Mar 22, 2024
32f28e1
Assume source is valid UTF8
overlookmotel Mar 22, 2024
068ca01
Comment
overlookmotel Mar 23, 2024
6a11a8f
`cloneType` function
overlookmotel Mar 26, 2024
2e7c0b2
`BindingPattern` deserializer no `...`
overlookmotel Mar 26, 2024
47cb102
Fix cargo-shear error
overlookmotel Mar 26, 2024
0f86395
Re-write deserializer generator
overlookmotel Mar 28, 2024
1aedbb4
Simplify creating buffer
overlookmotel Mar 28, 2024
4045edc
Move git ignore
overlookmotel Mar 29, 2024
80eff4c
Calculate niches in separate step
overlookmotel Mar 29, 2024
1ebdd70
Handle empty enums
overlookmotel Mar 29, 2024
0c873dc
Fix cargo-shear
overlookmotel Apr 21, 2024
9139cf2
Remove `Atom` custom code
overlookmotel Apr 21, 2024
fbcc11c
Comments
overlookmotel Apr 21, 2024
851bc1c
Fix: Add `#[ast_node]` on new AST types
overlookmotel Apr 22, 2024
79d2808
Remove explicit `Inspect` impls
overlookmotel Apr 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 6 additions & 85 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,98 +28,20 @@ concurrency:
cancel-in-progress: true

jobs:
benchmark:
name: Benchmark
runs-on: ubuntu-latest
strategy:
matrix:
# Run each benchmark in own job.
# Linter benchmark is by far the slowest, so split each fixture into own job.
component: [lexer, parser, transformer, semantic, minifier, codegen_sourcemap, sourcemap]
include:
- component: linter
fixture: 0
- component: linter
fixture: 1
- component: linter
fixture: 2
- component: linter
fixture: 3
- component: linter
fixture: 4
steps:
- name: Checkout Branch
uses: taiki-e/checkout-action@v1

- name: Install Rust Toolchain
uses: ./.github/actions/rustup
with:
shared-key: 'benchmark'
save-cache: ${{ github.ref_name == 'main' }}

- name: Install codspeed
uses: taiki-e/install-action@v2
with:
tool: cargo-codspeed

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: 20
registry-url: 'https://registry.npmjs.org'

- name: Start bench results interceptor server
working-directory: ./tasks/benchmark/codspeed
env:
COMPONENT: ${{ matrix.component }}
FIXTURE: ${{ matrix.fixture }}
run: |
corepack enable
pnpm install
node capture.mjs &

- name: Build benchmark
env:
RUSTFLAGS: "-C debuginfo=2 -C strip=none -g --cfg codspeed"
shell: bash
run: |
cargo build --release -p oxc_benchmark --bench ${{ matrix.component }} --features codspeed
mkdir -p target/codspeed/oxc_benchmark/
mv target/release/deps/${{ matrix.component }}-* target/codspeed/oxc_benchmark
rm -rf target/codspeed/oxc_benchmark/*.d

- name: Run benchmark
uses: CodSpeedHQ/action@v2
timeout-minutes: 30
env:
FIXTURE: ${{ matrix.fixture }}
with:
run: cargo codspeed run
# Dummy token for tokenless runs, to suppress logging hash of metadata JSON (see `upload.mjs`)
token: ${{ secrets.CODSPEED_TOKEN || 'dummy' }}
upload-url: http://localhost:${{ env.INTERCEPT_PORT }}/upload

- name: Upload bench data artefact
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.component }}${{ matrix.fixture }}
path: ${{ env.DATA_DIR }}
if-no-files-found: error
retention-days: 1

benchmark-napi:
name: Benchmark NAPI parser
runs-on: ubuntu-latest
if: false
steps:
- name: Checkout Branch
uses: taiki-e/checkout-action@v1
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: Install Rust Toolchain
uses: ./.github/actions/rustup
with:
shared-key: 'benchmark_napi'
save-cache: ${{ github.ref_name == 'main' }}
shared-key: benchmark_napi_ast_transfer
save-cache: true

- name: Install codspeed
uses: taiki-e/install-action@v2
Expand Down Expand Up @@ -186,8 +108,7 @@ jobs:

upload:
name: Upload benchmarks
# needs: [benchmark, benchmark-napi]
needs: [benchmark]
needs: benchmark-napi
runs-on: ubuntu-latest
steps:
- name: Checkout Branch
Expand Down
3 changes: 3 additions & 0 deletions .typos.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ extend-exclude = [
"**/*.snap",
"pnpm-lock.yaml",
"**/*/CHANGELOG.md",
"Cargo.toml",
]

[default.extend-words]
Expand All @@ -27,3 +28,5 @@ labeledby = "labeledby"

[default.extend-identifiers]
IIFEs = "IIFEs"
serName = "serName"
serValue = "serValue"
Loading