[test] Port tests in test/ to test/lit/basic/ #6160

aheejin · 2023-12-08T23:25:09Z

This ports all tests from test/ to test/lit/basic/. The set of commands and CHECK lines used are the same as the ones in #6159. Now we use lit to test these, this also deletes all .wast, .wast.from-wast, .wast.fromBinary, and
.wast.fromBinary.noDebugInfo files from test/ and all related test routines from the python scripts.

All CHECK lines are generated by update_lit_checks.py --all-items.

This also deletes these three multi-memory tests in test/lit/, because they seem to contain the same code with the ones in test/, which have been ported to test/lit/basic/ along with other tests.

test/lit/multi-memories-atomics64.wast
test/lit/multi-memories-basics.wast
test/lit/multi-memories-simd.wast

This also adds newlines between (funcs in case there are none to make CHECK lines easy to view, and removes some extra existing newlines here and there.

This ports all tests from `test/` to `test/lit/basic/`. The set of commands and `CHECK` lines used are the same as the ones in WebAssembly#6159. Now we use `lit` to test these, this also deletes all `.wast`, `.wast.from-wast`, `.wast.fromBinary`, and `.wast.fromBinary.noDebugInfo` files from `test/`. All tests lines are generated by `update_lit_checks.py --all-items`. This also deletes these three multi-memory tests in `test/lit/`, because they seem to contain the same code with the ones in `test/`, which have been ported to `test/lit/basic/` along with other tests. - `test/lit/multi-memories-atomics64.wast` - `test/lit/multi-memories-basics.wast` - `test/lit/multi-memories-simd.wast` This also adds newlines between `(func`s in case there are none, to make `CHECK` lines easy to view and removes some extra existing newlines here and there.

kripken · 2023-12-09T00:28:18Z

Looks like some test/externref.wast.* files are removed here without adding a new file to replace them - are those just unneeded files being removed? (I don't see any externref.wast base file for them.) Same for gc.wast, hello_world.wast, tail-call.wast, unit.wast.

aheejin · 2023-12-09T01:00:44Z

gc.wast was deleted in Delete test without use #5858 but not other .wast.* files.
anyref.wast was deleted in Add support for reference types proposal #2451 but not other .wast.* files. Those dangling .wast.* files were later renamed in bulk to externref.wast.* in Rename anyref to externref to match proposal change #2900.
I realized I only ported .wast. hello_world.wat and unit.wat have the .wat extension and haven't been ported. Will port them.
tail-call.wast was moved (and new functions added) to lit/tail-call.wast in Support subtyping in tail calls #4032 but other .wast.* files remained.

I also see some dangling tests in test/wasm2js which are causing wasm2js test failures. Probably will submit a separate PR to clean up some dangling tests...

aheejin · 2023-12-09T01:30:47Z

I didn't know this, but it looks wasm2js tests use some source files from test/ and test/spec/, and not only from test/wasm2js/:

binaryen/scripts/test/wasm2js.py

Line 51 in 1d615b3

for t in tests + spec_tests + wasm2js_tests:

I guess I just need to redirect them to test/lit/basic/...

aheejin · 2023-12-09T06:25:23Z

Removed test checking/updating routines for .wast.from-wast, .wast.fromBinary, and .wast.fromBinary.noDebugInfo.

Ported two wat tests (hello_world.wat and unit.wat) to test/lit/basic/, but left the original hello_world.wat in place in test/, because wasm_opt.py does some special tests with it and one of them requires the output to be the same as the input:

binaryen/scripts/test/wasm_opt.py

Lines 26 to 42 in 1d615b3

    
           for extra_args in [[], ['--no-validation']]: 
        
               wast = os.path.join(shared.options.binaryen_test, 'hello_world.wat') 
        
               shared.delete_from_orbit('a.wat') 
        
               out = 'a.wat' 
        
               cmd = shared.WASM_OPT + [wast, '-o', out, '-S', '-q'] + extra_args 
        
               support.run_command(cmd) 
        
               shared.fail_if_not_identical_to_file(open(out).read(), wast) 
        
           print('\n[ checking wasm-opt binary reading/writing... ]\n') 
        
           shutil.copyfile(os.path.join(shared.options.binaryen_test, 'hello_world.wat'), 'a.wat') 
        
           shared.delete_from_orbit('a.wasm') 
        
           shared.delete_from_orbit('b.wast') 
        
           support.run_command(shared.WASM_OPT + ['a.wat', '-o', 'a.wasm', '-q']) 
        
           assert open('a.wasm', 'rb').read()[0] == 0, 'we emit binary by default' 
        
           support.run_command(shared.WASM_OPT + ['a.wasm', '-o', 'b.wast', '-S', '-q']) 
        
           assert open('b.wast', 'rb').read()[0] != 0, 'we emit text with -S'

binaryen/scripts/test/wasm_opt.py

Lines 139 to 143 in 1d615b3

    
           print('\n[ checking wasm-opt -o notation... ]\n') 
        
           wast = os.path.join(shared.options.binaryen_test, 'hello_world.wat') 
        
           cmd = shared.WASM_OPT + [wast, '-o', 'a.wast', '-S'] 
        
           support.run_command(cmd) 
        
           open(wast, 'w').write(open('a.wast').read())

aheejin · 2023-12-10T21:36:27Z

The wasm2js failure in the CI will be fixed after #6164 lands.

aheejin · 2023-12-12T01:54:50Z

scripts/test/wasm_opt.py


 def update_wasm_opt_tests():
-    print('\n[ checking wasm-opt -o notation... ]\n')
+    print('\n[ updating wasm-opt -o notation... ]\n')


Drive-by fix: checking->updating in update_wasm_opt_tests messages

aheejin · 2023-12-12T01:55:30Z

scripts/test/wasm_opt.py

-
-        shared.minify_check(t)
-
-    print('\n[ checking wasm-opt debugInfo read-write... ]\n')


This line should've been removed in #6162

aheejin · 2023-12-12T18:48:25Z

Any more concerns on landing this PR?

We ported basic tests to `test/lit/basic/` in WebAssembly#6160, but comparing `CHECK` lines with the test code for long functions is not easy, even though it wouldn't necessarily be worse than the the separate files we used to have in `test/`. This slices `exception-handling.wast` into functions so that the `CHECK` lines are easy to check.

tlively

LGTM!

We ported basic tests to `test/lit/basic/` in WebAssembly#6160, but comparing `CHECK` lines with the test code for long functions is not easy, even though it wouldn't necessarily be worse than the the separate files we used to have in `test/`. This slices `exception-handling.wast` into functions so that the `CHECK` lines are easy to check.

We ported basic tests to `test/lit/basic/` in #6160, but comparing `CHECK` lines with the test code for long functions is not easy, even though it wouldn't necessarily be worse than the the separate files we used to have in `test/`. This slices `exception-handling.wast` into functions so that the `CHECK` lines are easy to check.

This ports all tests from `test/` to `test/lit/basic/`. The set of commands and `CHECK` lines used are the same as the ones in WebAssembly#6159. Now we use `lit` to test these, this also deletes all `.wast`, `.wast.from-wast`, `.wast.fromBinary`, and `.wast.fromBinary.noDebugInfo` files from `test/` and all related test routines from the python scripts. All `CHECK` lines are generated by `update_lit_checks.py --all-items`. This also deletes these three multi-memory tests in `test/lit/`, because they seem to contain the same code with the ones in `test/`, which have been ported to `test/lit/basic/` along with other tests. - `test/lit/multi-memories-atomics64.wast` - `test/lit/multi-memories-basics.wast` - `test/lit/multi-memories-simd.wast` This also adds newlines between `(func`s in case there are none to make `CHECK` lines easy to view, and removes some extra existing newlines here and there.

We ported basic tests to `test/lit/basic/` in WebAssembly#6160, but comparing `CHECK` lines with the test code for long functions is not easy, even though it wouldn't necessarily be worse than the the separate files we used to have in `test/`. This slices `exception-handling.wast` into functions so that the `CHECK` lines are easy to check.

aheejin requested review from kripken, ashleynh and tlively December 8, 2023 23:25

aheejin mentioned this pull request Dec 9, 2023

[EH] Add exnref type back #6149

Merged

Change a hard-coded path in check.py

fbcf48f

aheejin added 2 commits December 8, 2023 22:06

Fix some newlines and port wat tests too

2ef9b79

Remove wasm-opt/binary tests from test scripts

428d2ae

wasm2js script fix

69ab50b

aheejin requested a review from sbc100 December 10, 2023 20:30

Factor out assert test fix to WebAssembly#6164

ec41916

kripken approved these changes Dec 11, 2023

View reviewed changes

Merge branch 'main' into port_basic_tests

51a1b9f

aheejin commented Dec 12, 2023

View reviewed changes

aheejin added 2 commits December 11, 2023 19:20

Merge branch 'main' into port_basic_tests

6e0db3c

Remove a newline

11e0fbb

kripken approved these changes Dec 12, 2023

View reviewed changes

tlively approved these changes Dec 13, 2023

View reviewed changes

Merge branch 'main' into port_basic_tests

ced2897

aheejin merged commit 0024c8b into WebAssembly:main Dec 13, 2023
14 checks passed

aheejin deleted the port_basic_tests branch December 13, 2023 18:53

aheejin mentioned this pull request Dec 13, 2023

[EH][test] Slice test code into different functions #6177

Merged

aheejin mentioned this pull request Dec 16, 2023

Typed continuations: resume instructions #6083

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[test] Port tests in test/ to test/lit/basic/ #6160

[test] Port tests in test/ to test/lit/basic/ #6160

aheejin commented Dec 8, 2023 •

edited

Loading

kripken commented Dec 9, 2023

aheejin commented Dec 9, 2023 •

edited

Loading

aheejin commented Dec 9, 2023

aheejin commented Dec 9, 2023

aheejin commented Dec 10, 2023

aheejin Dec 12, 2023

aheejin Dec 12, 2023

aheejin commented Dec 12, 2023

tlively left a comment


		shared.minify_check(t)

		print('\n[ checking wasm-opt debugInfo read-write... ]\n')

[test] Port tests in test/ to test/lit/basic/ #6160

[test] Port tests in test/ to test/lit/basic/ #6160

Conversation

aheejin commented Dec 8, 2023 • edited Loading

kripken commented Dec 9, 2023

aheejin commented Dec 9, 2023 • edited Loading

aheejin commented Dec 9, 2023

aheejin commented Dec 9, 2023

aheejin commented Dec 10, 2023

aheejin Dec 12, 2023

Choose a reason for hiding this comment

aheejin Dec 12, 2023

Choose a reason for hiding this comment

aheejin commented Dec 12, 2023

tlively left a comment

Choose a reason for hiding this comment

aheejin commented Dec 8, 2023 •

edited

Loading

aheejin commented Dec 9, 2023 •

edited

Loading