-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tidy up preprocessor and parser #12
Commits on Jul 2, 2023
-
Fix typos in preprocessor tests
These don’t really affect the preprocessor because it doesn’t care about the details, but it’s less confusing if the modules in the tests are at least syntactically valid.
Configuration menu - View commit details
-
Copy full SHA for b6f09f5 - Browse repository at this point
Copy the full SHA b6f09f5View commit details
Commits on Jul 4, 2023
-
Extract #assert_preprocess_module_fields test helper
Since almost all of the preprocessor tests are for fields within a single module body, we can reduce boilerplate (and indentation levels) by introducing an assertion which automatically wraps its arguments in `(module …)`.
Configuration menu - View commit details
-
Copy full SHA for 3cf39b6 - Browse repository at this point
Copy the full SHA 3cf39b6View commit details -
Update preprocessor tests to not depend upon type use abbreviation
Most of the preprocessor tests use syntax which relies upon the ability to omit an explicit type use [0] and have it automatically inserted when abbreviations are expanded. To prevent these tests from breaking when we implement desugaring of type use abbreviations we need to update them to contain full type uses which are unaffected by the expansion of that abbreviation. When adding an unabbreviated type use, I’ve removed the corresponding inline declarations unless they’re the focus of the test. [0] https://webassembly.github.io/spec/core/text/modules.html#abbreviations
Configuration menu - View commit details
-
Copy full SHA for 207c4ad - Browse repository at this point
Copy the full SHA 207c4adView commit details
Commits on Jul 5, 2023
-
Pretty print S-expressions when a preprocessor test fails
This makes it easier to see what’s gone wrong.
Configuration menu - View commit details
-
Copy full SHA for 27c94c4 - Browse repository at this point
Copy the full SHA 27c94c4View commit details -
Preprocess instructions inside global definitions
As with `offset` and element expressions, the test is demonstrating desugaring of instructions in an invalid program because that’s the correct behaviour for the parsing phase regardless of validation. In practice only constant instructions [0] may appear inside a global definition [1] but we have no way to demonstrate desugaring of constant instructions yet. [0] https://webassembly.github.io/spec/core/valid/instructions.html#valid-constant [1] https://webassembly.github.io/spec/core/valid/modules.html#globals
Configuration menu - View commit details
-
Copy full SHA for dc2465b - Browse repository at this point
Copy the full SHA dc2465bView commit details -
Recursively preprocess expanded inline import
We need to desugar an inline import after expanding it because its description can contain other abbreviations (e.g. multiple anonymous parameters and results in the case of function imports). I overlooked this earlier because I didn’t write a test to cover it and presumably the official WebAssembly test suite doesn’t cover it either, so no tests failed when I removed inline import support from the parser.
Configuration menu - View commit details
-
Copy full SHA for ea67cc0 - Browse repository at this point
Copy the full SHA ea67cc0View commit details -
Make preprocessing for
export
andstart
fields explicitThese are the only two remaining kinds of field and neither supports any abbreviations, so we can list them as an intentional no-op case and have all field kinds explicitly represented.
Configuration menu - View commit details
-
Copy full SHA for 41f065c - Browse repository at this point
Copy the full SHA 41f065cView commit details -
Decouple reading from result generation everywhere in preprocessor
This separation exists in most places in the preprocessor, so for the sake of stylistic consistency we should do it everywhere.
Configuration menu - View commit details
-
Copy full SHA for 02663dc - Browse repository at this point
Copy the full SHA 02663dcView commit details -
Rename module field parsing methods to match language spec and prepro…
…cessor There’s no reason for these to be different, and the longer names are clearer.
Configuration menu - View commit details
-
Copy full SHA for 7ed0b57 - Browse repository at this point
Copy the full SHA 7ed0b57View commit details -
Extract #fresh_id helper in preprocessor
It’s still the wrong way of doing it, but at least now it’s only wrong once.
Configuration menu - View commit details
-
Copy full SHA for b2d3ea8 - Browse repository at this point
Copy the full SHA b2d3ea8View commit details
Commits on Jul 6, 2023
-
Extract ReadOptionalId helper module from preprocessor and parser
This logic is repeated in a lot of places, so let’s extract and reuse it. IDs are essentially always optional so I’ve chosen to write a single method which unconditionally tries to read an ID (and may return nil) rather than a pair of #can_read_id?/#read_id methods. That does mean we have to check for `id.nil?` in a couple of places rather than asking a dedicated predicate, but it’s rare enough that I think this trade-off is the right one.
Configuration menu - View commit details
-
Copy full SHA for 4f3c2e9 - Browse repository at this point
Copy the full SHA 4f3c2e9View commit details -
Use pattern matching to check for index when parsing
br_table
instr……uction This is how we do it everywhere else, and there’s no reason for it to be different here. The pattern match also has the advantage of tolerating non-string values (e.g. arrays), which makes the #can_read_list? call unnecessary. Removing the numbered parameter changes the block’s arity, so we need to add an explicit dummy parameter to keep #repeatedly happy. In hindsight I don’t think the `until:` API is a good idea so I’m going to change that soon.
Configuration menu - View commit details
-
Copy full SHA for 08436fe - Browse repository at this point
Copy the full SHA 08436feView commit details -
Extract ReadIndex helper module from AST parser
This provides a nice expressive interface for dealing with indexes, moves a complicated regexp out of the parser, and puts the implementation in a helper where the preprocessor will be able to share it in future.
Configuration menu - View commit details
-
Copy full SHA for adf38a6 - Browse repository at this point
Copy the full SHA adf38a6View commit details
Commits on Jul 16, 2023
-
Add
?:
to regular expressions without named groupsIn 8ea2aa0 I removed unnecessary `?:` from regular expressions which already didn’t capture unnamed groups (because they also contained named ones), but I neglected to check whether other regular expressions were unintentionally capturing unnamed groups, so I’m fixing that now. Although this adds more syntactic noise to the regular expressions, I think our intent is clearer if we only capture groups which we intend to use, so I’m prepared to pay the price.
Configuration menu - View commit details
-
Copy full SHA for 18f1b51 - Browse repository at this point
Copy the full SHA 18f1b51View commit details -
Clean up action read in #parse_assert_return and #parse_assert_trap
The pattern match will raise an exception if it fails, so there’s no need to do it ourselves.
Configuration menu - View commit details
-
Copy full SHA for 9483690 - Browse repository at this point
Copy the full SHA 9483690View commit details -
Use #parse_typeuse when parsing
call_indirect
instructionThe grammar says a type use appears here [0] so we should be reusing all of the appropriate parsing machinery. This creates the opportunity to check that no named parameters appear in the type use, which we’ll do in the next commit. [0] https://webassembly.github.io/spec/core/text/instructions.html#text-call-indirect
Configuration menu - View commit details
-
Copy full SHA for c770866 - Browse repository at this point
Copy the full SHA c770866View commit details -
Check that no named parameters appear in
call_indirect
type useWe already do this in #parse_blocktype because of its side condition which requires a type use to contain only unnamed entries [0]. The same side condition applies to `call_indirect`’s type use [1] so we should perform the same check. [0] https://webassembly.github.io/spec/core/text/instructions.html#text-instr-block [1] https://webassembly.github.io/spec/core/text/instructions.html#text-call-indirect
Configuration menu - View commit details
-
Copy full SHA for 06c71da - Browse repository at this point
Copy the full SHA 06c71daView commit details -
Use #parse_results when parsing
select
instructionThe grammar says zero or more results appear here [0] so we should be reusing the appropriate parsing machinery. [0] https://webassembly.github.io/spec/core/text/instructions.html#parametric-instructions
Configuration menu - View commit details
-
Copy full SHA for 499b1e1 - Browse repository at this point
Copy the full SHA 499b1e1View commit details -
Use #parse_index when parsing
table.init
instructionThese are indexes [0], so we should be reusing the index-parsing machinery. [0] https://webassembly.github.io/spec/core/text/instructions.html#table-instructions
Configuration menu - View commit details
-
Copy full SHA for af92d91 - Browse repository at this point
Copy the full SHA af92d91View commit details -
Use Kernel#loop in #repeatedly helper
This’ll allow the loop to be terminated by the `StopIteration` exception instead of needing to support custom conditions through arguments.
Configuration menu - View commit details
-
Copy full SHA for f3681f3 - Browse repository at this point
Copy the full SHA f3681f3View commit details -
Replace
until:
with raisingStopIteration
in #repeatedly callersIt’s admittedly a bit “exceptions for control flow”, but this way of using `StopIteration` is idiomatic in Ruby so I think giving the caller arbitrary control over loop termination is easier to understand than relying upon a mystery `until:` argument. I’ve resisted the temptation to push `raise StopIteration` further down the call stack (into e.g. #read_list, or even #read) to avoid spooky action at a distance; I think the logic is easiest to understand when `raise StopIteration` appears syntactically within the #repeatedly block. I’ve also chosen a guard-style raise (`raise … if …`) rather than putting the whole body of the block inside a conditional (`if … else raise … end`) because the former puts the `raise` lexically adjacent to the #repeatedly call instead of pushing it further away (e.g. inside the `else` clause) which should improve clarity. This choice also helps to keep the indentation levels under control.
Configuration menu - View commit details
-
Copy full SHA for f87fbb2 - Browse repository at this point
Copy the full SHA f87fbb2View commit details -
Remove unused
until:
argument from #repeatedly helperWe always use `StopIteration` now.
Configuration menu - View commit details
-
Copy full SHA for 347c635 - Browse repository at this point
Copy the full SHA 347c635View commit details -
Use #repeatedly helper instead of
while
/until
in preprocessor and…… parser This avoids us having to do the concatenating ourselves. Again I’ve chosen guard-style `raise StopIteration if …` clauses to keep them adjacent to the #repeatedly call for clarity’s sake.
Configuration menu - View commit details
-
Copy full SHA for bff8044 - Browse repository at this point
Copy the full SHA bff8044View commit details -
Disable YJIT when running tests
I’d like to use destructuring with optional block parameters in the implementation of #unzip_pairs, but there’s a bug in YJIT [0] which prevents this from working. The easiest solution for now is to turn YJIT off. [0] Shopify/yjit#313
Configuration menu - View commit details
-
Copy full SHA for 192014b - Browse repository at this point
Copy the full SHA 192014bView commit details -
Use optional block parameters in #unzip_pairs
When `pairs` is empty then `pairs.transpose` is `[]`, which gets destructured into a pair of nils for the two block parameters. We can replace the nils in this case by defaulting each parameter to the empty array.
Configuration menu - View commit details
-
Copy full SHA for dd5488c - Browse repository at this point
Copy the full SHA dd5488cView commit details -
Push #unzip_pairs into #parse_parameters and #parse_locals
The only uses of #unzip_pairs are to unzip the results of these methods, so we might as well have them return an unzipped result.
Configuration menu - View commit details
-
Copy full SHA for 505caa5 - Browse repository at this point
Copy the full SHA 505caa5View commit details -
Use #unzip_pairs in #parse_results
This was the only caller of #parse_declarations which didn’t unzip its result with #unzip_pairs. By doing it consistently across all callers we create the opportunity to have #parse_declarations return unzipped results directly, which we’ll do in the next commit.
Configuration menu - View commit details
-
Copy full SHA for 53ecd0d - Browse repository at this point
Copy the full SHA 53ecd0dView commit details -
Push #unzip_pairs into #parse_declarations
The only uses of #unzip_pairs are to unzip the result of the #parse_declarations method, so we might as well have it return an unzipped result.
Configuration menu - View commit details
-
Copy full SHA for c98e965 - Browse repository at this point
Copy the full SHA c98e965View commit details -
Inline #unzip_pairs into #parse_declarations
This is now the only caller of #unzip_pairs, so we can safely build it into #parse_declarations directly.
Configuration menu - View commit details
-
Copy full SHA for b951678 - Browse repository at this point
Copy the full SHA b951678View commit details -
Support multiple memories in AST parser
Validation constrains modules to at most one memory [0] but the parser doesn’t need to worry about that. It creates more regularity in the parser to support arbitrarily many memories (just like functions, tables, globals etc) and let validation deal with any problems. [0] https://webassembly.github.io/spec/core/valid/modules.html#valid-module
Configuration menu - View commit details
-
Copy full SHA for f253bb0 - Browse repository at this point
Copy the full SHA f253bb0View commit details -
Initialise all module field vectors together in AST parser
Now that these are all arrays (except for `start`) we can consolidate the setup of their initial values.
Configuration menu - View commit details
-
Copy full SHA for 6df4e7a - Browse repository at this point
Copy the full SHA 6df4e7aView commit details -
Consolidate references to singleton memory in interpreter
I’d like to support multiple memories here to match the AST parser, and that change will be easier if we assign `current_module.memory` to a local variable instead of repeatedly using it inline.
Configuration menu - View commit details
-
Copy full SHA for 17157a4 - Browse repository at this point
Copy the full SHA 17157a4View commit details -
Support multiple runtime memories in interpreter
Validation won’t actually allow multiple memories [0] but it makes the implementation more regular, and more like the formal WebAssembly language specification, if we assume they’re possible. All of the memory instructions implicitly operate on memory index zero [1] so we’ve now made that explicit in their implementation. The #initialise_memories method no longer raises an error if the memory index is non-zero; this check should’ve been happening during validation anyway (which we’re not doing yet) so losing it doesn’t matter. [0] https://webassembly.github.io/spec/core/valid/modules.html#valid-module [1] https://webassembly.github.io/spec/core/syntax/instructions.html#memory-instructions
Configuration menu - View commit details
-
Copy full SHA for 1ddfef3 - Browse repository at this point
Copy the full SHA 1ddfef3View commit details -
Extract #allocate_instances helper in interpreter
Now that modules have multiple memories, we can use the same approach for all relevant fields during allocation [0]. This makes it clear that exports are different and shouldn’t be “built” in the same way; we could’ve already seen this from the definition of module allocation [1]. I’m not going to get distracted by making any more changes to the interpreter at the moment, but we should come back to this in future. [0] https://webassembly.github.io/spec/core/exec/modules.html#allocation [1] https://webassembly.github.io/spec/core/exec/modules.html#alloc-module
Configuration menu - View commit details
-
Copy full SHA for 9042b94 - Browse repository at this point
Copy the full SHA 9042b94View commit details