Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge with upstream #171

Merged
merged 15 commits into from
May 15, 2024
Merged

Merge with upstream #171

merged 15 commits into from
May 15, 2024

Conversation

dhil
Copy link
Member

@dhil dhil commented May 15, 2024

No description provided.

ulan and others added 15 commits May 14, 2024 14:22
The toml file specifies version `0.4.1` instead of `0.4.2`.

Using version `0.4.1` produces a compile error:

```
error[E0432]: unresolved import `mach2::ndr`
  --> external/crate_index__wasmtime-runtime-20.0.2/src/sys/unix/machports.rs:44:12
   |
44 | use mach2::ndr::*;
   |            ^^^ could not find `ndr` in `mach2`
```

That's because `ndr` was added in version `0.4.2`.

Note that the lock file specifies version `0.4.2` which explains
why this error doesn't happen normally.
…lliance#8610)

This commit aims to address bytecodealliance#8607 by dynamically determining whether the
pooling allocator should be used rather than unconditionally using it.
It looks like some systems don't have enough virtual memory to support
the default configuration settings so this should help `wasmtime serve`
work on those systems.

Closes bytecodealliance#8607
* wasmtime: Make table lazy-init configurable

Lazy initialization of tables has trade-offs that we haven't explored in
a while. Making it configurable makes it easier to test the effects of
these trade-offs on a variety of WebAssembly programs, and allows
embedders to decide whether the trade-offs are worth-while for their use
cases.

* Review comments
* Allow env/args/preopens to exceed 64k in size

This commit fixes an issue with the wasip1 adapter published with
Wasmtime which current limits the size of environment variables,
arguments, and preopens to not exceed 64k. This bug comes from the fact
that we more-or-less forgot to account for this when designing the
adapter initially. The adapter allocates a single WebAssembly page for
itself but does not have a means of allocating much more than that. It's
technically possible to continue to call `memory.grow` or possibly
`cabi_realloc` from the original main module but it's pretty awkward.

The solution in this commit is to take an alternative approach to how
these properties are all processed. Previously arguments/env
vars/preopens were all allocated once within the adapter and stored
statically. This means that after startup they're copied from where they
reside in-memory, but the downside is that we have to have enough memory
to hold everything. This commit instead tries to "stream" the items so
they're never held entirely within the adapter itself.

The general idea in this commit is to use the "align" parameter to
`cabi_import_realloc` to figure out what's being allocated and route the
allocation to the destination. For example an allocation with alignment
1 is a string and can go directly into a user-supplied pointer where an
allocation with alignment 4 is a pointer-based allocation which must be
stored within the adapter, but only temporarily.

With this redesign it's now possible to have the sum total of
args/envs/preopens to exceed 64k. The new limitation is that the
max-length string plus size of the max length of these arrays must be
less than 64k. This should be a more reasonable limit than before where
any one individual argument/env var is unlikely to exceed 64k (or get
close).

Closes bytecodealliance#8556

* Comment descriptors are closed

* Update crates/wasi-preview1-component-adapter/src/descriptors.rs

Co-authored-by: Trevor Elliott <awesomelyawesome@gmail.com>

* Turn down process limits for macOS

Looks like a 1M env block is a bit too large.

---------

Co-authored-by: Trevor Elliott <awesomelyawesome@gmail.com>
* Test that wasi file streams can handle read(0)

* Zero-sized reads don't fail for file streams

* Accidentally removed the `read(0)` when refactoring the test
* Change `Tunables::static_memory_bound` to bytes

This commit changes the wasm-page-sized `static_memory_bound` field to
instead being a byte-defined unit rather than a page-defined unit. To
accomplish this the field is renamed to `static_memory_reservation` and
all references are updated. This builds on the support from bytecodealliance#8608 to
remove another page-based variable from the internals of Wasmtime.

* Fix tests
…odealliance#8593)

The latter is what Wasmtime uses today but it pulls in parsers for all
object formats supported by `object`. In the context of Wasmtime,
however, we know that all objects produced are 64-bit ELF files so
there's no need to pull in, for example, a COFF parser as that'll always
return an error anyway. This commit switches uses of the `object::File`
convenience to `ElfFile64` instead.
…e#8609)

This commit builds on the support from bytecodealliance#8448 to remove all blanket impls
from the WASI crates and instead replace them with concrete impls. This
is slightly functionally different from before where impls are now on
trait objects meaning dynamic dispatch is involved where previously
dynamic dispatch was used. That being said the perf hit here is expected
to be negligible-to-nonexistent since the implementations are large
enough that the dynamic dispatch won't be the hot path.

The motivations for this commit are:

* Removes the need for an odd `skip_mut_forwarding_impls` option - but
  this'll be left for a bit in case others need it.
* Improves incremental compile time of these crates where the crates
  themselves now contain all object code for all of WASI instead of
  forcing the final consume to codegen everything (although there's
  still a significant amount monomorphized).
* Improves future compatibility with refactorings of
  bindgen-generated-traits and such because blanket impls are pretty
  hard to work around where concrete impls are easier to reason about
  (and document).
* prtest:full

* Disable some extremely slow MIRI tests
)

Consumption of non-allocatable operands was added in bytecodealliance#5253 and bytecodealliance#5132,
and removed in bytecodealliance#8524 and following PRs. Now they are not only ignored by
regalloc2, but the placeholders that it leaves in the allocation results
are also ignored by Cranelift. So let's stop adding them to the operands
list entirely.
Fixes an accidental fuzz regression from bytecodealliance#8590 where error messages were
changed slightly.
…#8619)

* prtest:full

* CI: Split DWARF debugging test out into its own job
The egraph pass was already doing this, when it ran, and it never adds
any aliases. So do it slightly earlier and unconditionally, and avoid
needing to resolve any aliases during lowering.
@dhil dhil merged commit 9c9264c into wasmfx:main May 15, 2024
27 checks passed
@dhil dhil deleted the wasmfx-merge branch May 15, 2024 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants