Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI test job using LLVM's Address Sanitizer #891

Merged
merged 2 commits into from
Jan 15, 2024
Merged

Conversation

Robbepop
Copy link
Member

Closes #890.

@paritytech-cicd-pr
Copy link

BENCHMARKS

NATIVEWASMTIME
BENCHMARKMASTERPRDIFFMASTERPRDIFFWASMTIME OVERHEAD
execute/
br_table
1.63ms 1.59ms 🟢 -2.53% 1.29ms 1.29ms ⚪ -0.28% 🟢 -19%
execute/
call/host/1
51.56µs 50.51µs ⚪ -2.03% 57.76µs 57.76µs ⚪ 0.00% 🟢 14%
execute/
call/rec
170.97µs 169.89µs ⚪ -0.63% 289.85µs 288.89µs ⚪ -0.33% 🟡 70%
execute/
count_until
5.51ms 5.51ms ⚪ 0.07% 7.28ms 7.26ms ⚪ -0.32% 🟢 32%
execute/
divrem
6.40ms 6.16ms 🟢 -3.80% 6.26ms 6.25ms ⚪ -0.17% 🟢 2%
execute/
factorial/iter
236.89µs 229.40µs 🟢 -3.16% 273.53µs 271.45µs ⚪ -0.76% 🟢 18%
execute/
factorial/rec
681.15µs 698.73µs 🔴 2.58% 1.09ms 1.09ms ⚪ 0.01% 🟡 56%
execute/
fibonacci/iter
1.33ms 1.33ms ⚪ -0.04% 1.14ms 1.14ms ⚪ -0.04% 🟢 -14%
execute/
fibonacci/rec
5.83ms 5.84ms ⚪ 0.02% 10.87ms 10.95ms ⚪ 0.79% 🟡 88%
execute/
fibonacci/tail
1.25ms 1.25ms ⚪ 0.03% 4.39ms 4.32ms 🟢 -1.60% 🔴 246%
execute/
fuse
7.04ms 7.04ms ⚪ 0.04% 11.13ms 11.13ms ⚪ 0.04% 🟡 58%
execute/
global/bump
1.32ms 1.32ms ⚪ -0.01% 1.41ms 1.41ms ⚪ -0.09% 🟢 7%
execute/
global/get_const
481.94µs 481.82µs ⚪ -0.03% 723.35µs 724.11µs ⚪ 0.10% 🟢 50%
execute/
is_even/rec
1.12ms 1.10ms ⚪ -2.15% 1.84ms 1.84ms ⚪ -0.18% 🟡 68%
execute/
memory/fill_bytes
1.00ms 1.00ms ⚪ 0.00% 1.20ms 1.20ms ⚪ -0.18% 🟢 19%
execute/
memory/sum_bytes
1.14ms 1.14ms ⚪ 0.24% 1.20ms 1.20ms ⚪ 0.13% 🟢 5%
execute/
memory/vec_add
2.95ms 2.95ms ⚪ 0.08% 3.53ms 3.52ms ⚪ -0.43% 🟢 19%
execute/
recursive_scan
189.46µs 189.73µs ⚪ 0.14% 314.32µs 313.04µs ⚪ -0.41% 🟡 65%
execute/
recursive_trap
16.29µs 15.85µs 🟢 -2.74% 27.29µs 27.88µs 🔴 2.15% 🟡 76%
execute/
regex_redux
602.05µs 605.33µs ⚪ 0.54% 1.07ms 1.05ms 🟢 -1.81% 🟡 73%
execute/
rev_complement
440.84µs 441.23µs ⚪ 0.09% 620.60µs 622.19µs ⚪ 0.26% 🟢 41%
execute/
tiny_keccak
348.56µs 348.45µs ⚪ -0.03% 329.03µs 329.36µs ⚪ 0.10% 🟢 -5%
execute/
trunc_f2i
637.99µs 639.58µs ⚪ 0.25% 945.47µs 947.48µs ⚪ 0.21% 🟢 48%
instantiate/
wasm_kernel
55.57µs 52.75µs 🟢 -5.08% 55.46µs 56.81µs ⚪ 2.44% 🟢 8%
overhead/
call/typed/0
1.25ms 1.23ms ⚪ -1.08% 862.04µs 840.11µs 🟢 -2.54% 🟢 -32%
overhead/
call/typed/16
1.65ms 1.65ms ⚪ 0.17% 2.43ms 2.53ms 🔴 4.17% 🟡 53%
overhead/
call/untyped/0
1.59ms 1.58ms ⚪ -1.07% 1.15ms 1.17ms ⚪ 1.12% 🟢 -26%
overhead/
call/untyped/16
2.45ms 2.46ms ⚪ 0.63% 4.32ms 4.27ms ⚪ -1.00% 🟡 73%
translate/
bz2/checked/eager/default
1.34ms 1.34ms ⚪ -0.14% 2.43ms 2.45ms ⚪ 0.81% 🟡 82%
translate/
bz2/checked/eager/fuel
1.45ms 1.45ms ⚪ -0.29% 2.65ms 2.65ms ⚪ 0.03% 🟡 83%
translate/
bz2/checked/lazy-translation/default
543.30µs 544.98µs ⚪ 0.31% 972.86µs 979.64µs ⚪ 0.70% 🟡 80%
translate/
bz2/checked/lazy/default
36.71µs 36.67µs ⚪ -0.12% 46.47µs 46.68µs ⚪ 0.46% 🟢 27%
translate/
bz2/unchecked/eager/default
1.09ms 1.09ms ⚪ 0.34% 1.90ms 1.91ms ⚪ 0.66% 🟡 75%
translate/
erc1155/checked/eager/default
281.66µs 281.40µs ⚪ -0.09% 478.02µs 483.04µs ⚪ 1.05% 🟡 72%
translate/
erc1155/checked/eager/fuel
301.47µs 301.20µs ⚪ -0.09% 518.26µs 520.30µs ⚪ 0.39% 🟡 73%
translate/
erc1155/checked/lazy-translation/default
127.84µs 128.40µs ⚪ 0.43% 214.58µs 219.27µs 🔴 2.19% 🟡 71%
translate/
erc1155/checked/lazy/default
24.54µs 25.96µs 🔴 5.77% 34.21µs 34.15µs ⚪ -0.17% 🟢 32%
translate/
erc1155/unchecked/eager/default
231.52µs 231.36µs ⚪ -0.07% 384.11µs 380.38µs ⚪ -0.97% 🟡 64%
translate/
erc20/checked/eager/default
136.65µs 136.66µs ⚪ 0.00% 228.93µs 233.00µs 🔴 1.78% 🟡 71%
translate/
erc20/checked/eager/fuel
143.80µs 144.52µs ⚪ 0.50% 242.86µs 245.72µs ⚪ 1.18% 🟡 70%
translate/
erc20/checked/lazy-translation/default
65.05µs 65.88µs ⚪ 1.27% 108.42µs 111.21µs 🔴 2.57% 🟡 69%
translate/
erc20/checked/lazy/default
18.88µs 19.23µs 🔴 1.85% 25.32µs 25.93µs 🔴 2.41% 🟢 35%
translate/
erc20/unchecked/eager/default
111.70µs 112.22µs ⚪ 0.47% 180.06µs 182.41µs ⚪ 1.30% 🟡 63%
translate/
erc721/checked/eager/default
194.26µs 193.98µs ⚪ -0.14% 332.09µs 336.11µs ⚪ 1.21% 🟡 73%
translate/
erc721/checked/eager/fuel
203.30µs 204.54µs ⚪ 0.61% 350.52µs 353.92µs ⚪ 0.97% 🟡 73%
translate/
erc721/checked/lazy-translation/default
91.45µs 90.88µs ⚪ -0.62% 153.96µs 156.34µs 🔴 1.55% 🟡 72%
translate/
erc721/checked/lazy/default
23.51µs 23.40µs ⚪ -0.47% 29.33µs 30.07µs 🔴 2.53% 🟢 28%
translate/
erc721/unchecked/eager/default
156.79µs 157.41µs ⚪ 0.40% 255.05µs 258.92µs 🔴 1.52% 🟡 64%
translate/
pulldown_cmark/checked/eager/default
3.70ms 3.71ms ⚪ 0.30% 6.21ms 6.28ms ⚪ 1.05% 🟡 69%
translate/
pulldown_cmark/checked/eager/fuel
3.97ms 3.99ms ⚪ 0.47% 6.73ms 6.76ms ⚪ 0.55% 🟡 70%
translate/
pulldown_cmark/checked/lazy-translation/default
1.55ms 1.54ms ⚪ -0.26% 2.57ms 2.62ms 🔴 1.91% 🟡 70%
translate/
pulldown_cmark/checked/lazy/default
254.49µs 247.20µs 🟢 -2.86% 248.40µs 259.26µs 🔴 4.37% 🟢 5%
translate/
pulldown_cmark/unchecked/eager/default
3.06ms 3.10ms ⚪ 1.13% 4.93ms 4.95ms ⚪ 0.52% 🟡 60%
translate/
spidermonkey/checked/eager/default
77.28ms 77.31ms ⚪ 0.03% 134.88ms 136.09ms ⚪ 0.89% 🟡 76%
translate/
spidermonkey/checked/eager/fuel
83.49ms 83.79ms ⚪ 0.36% 149.89ms 146.69ms ⚪ -2.13% 🟡 75%
translate/
spidermonkey/checked/lazy-translation/default
32.86ms 32.90ms ⚪ 0.12% 56.43ms 57.31ms 🔴 1.57% 🟡 74%
translate/
spidermonkey/checked/lazy/default
3.59ms 3.67ms ⚪ 2.37% 3.95ms 4.12ms 🔴 4.29% 🟢 12%
translate/
spidermonkey/unchecked/eager/default
64.09ms 64.18ms ⚪ 0.14% 106.32ms 106.92ms ⚪ 0.56% 🟡 67%
translate/
wasm_kernel/checked/eager/default
5.15ms 5.14ms ⚪ -0.04% 8.78ms 8.99ms 🔴 2.31% 🟡 75%
translate/
wasm_kernel/checked/eager/fuel
5.30ms 5.28ms ⚪ -0.32% 9.27ms 9.39ms ⚪ 1.27% 🟡 78%
translate/
wasm_kernel/checked/lazy-translation/default
2.41ms 2.42ms ⚪ 0.09% 4.03ms 4.12ms 🔴 2.44% 🟡 71%
translate/
wasm_kernel/checked/lazy/default
425.79µs 423.07µs ⚪ -0.64% 501.29µs 512.76µs 🔴 2.29% 🟢 21%
translate/
wasm_kernel/unchecked/eager/default
4.16ms 4.19ms ⚪ 0.77% 6.79ms 6.88ms 🔴 1.33% 🟡 64%

Link to pipeline

@Robbepop Robbepop merged commit e60da49 into master Jan 15, 2024
20 of 21 checks passed
@Robbepop Robbepop deleted the rf-asan-ci-testing branch January 15, 2024 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run CI tests using LLVM's address sanitizer
2 participants