Wasmer 3 🔥 #1674

webmaster128 · 2023-04-27T10:03:15Z

No description provided.

Co-authored-by: Simon Warta <2603011+webmaster128@users.noreply.github.com>

W3 new store #2

webmaster128 · 2023-05-23T16:02:54Z

~~The CI failure will be fixed here: #1691~~ Done

maurolacy

Partial review. Will finish with imports.rs and instance.rs next.

maurolacy · 2023-05-24T08:20:38Z

.circleci/config.yml

@@ -85,6 +85,7 @@ workflows:
                - /^[0-9]+\.[0-9]+$/
                # Add your branch here if benchmarking matters to your work
                - fix-benchmarking
+                - w3


Can now be removed perhaps?

packages/vm/examples/module_size.sh

packages/vm/src/environment.rs

maurolacy · 2023-05-24T08:28:35Z

packages/vm/src/environment.rs

+    // Creates a MemoryView.
+    // This mut be short living and not be used after the memory was grown.
+    pub fn memory<'a>(&self, store: &'a mut impl AsStoreMut) -> MemoryView<'a> {
+        self.memory
+            .as_ref()
+            .expect("Memory is not set. This is a bug in the lifecycle.")
+            .view(store)


Is this for performance or what? Just curious.

It creates a MemoryView as designed by the new Wasmer API:

/// A WebAssembly `memory` view. /// /// A memory view is used to read and write to the linear memory. /// /// After a memory is grown a view must not be used anymore. Views are /// created using the Memory.grow() method. #[derive(Debug)] pub struct MemoryView<'a>(pub(crate) memory_view_impl::MemoryView<'a>);

It seems like the Memory itself cannot be used directly:

#[derive(Debug, Clone, PartialEq)] pub struct Memory(pub(crate) memory_impl::Memory); impl Memory { // ... /// Creates a view into the memory that then allows for /// read and write pub fn view<'a>(&self, store: &'a impl AsStoreRef) -> MemoryView<'a> { MemoryView::new(self, store) } // ... }

maurolacy · 2023-05-24T08:33:34Z

packages/vm/src/memory.rs

+    fn zero_padding_bytes(&self, _bytes: &mut [MaybeUninit<u8>]) {
+        // The size of Region is exactly 3x4=12 bytes with no padding.
+        // The `size_of::<Region>()` test below ensures that.
+        // So we do not need to zero any bytes here.
+    }


A 'comment function' 🙂.

packages/vm/src/memory.rs

maurolacy · 2023-05-24T08:43:03Z

packages/vm/src/wasm_backend/gatekeeper.rs

-            | Operator::F32x4Fms
-            | Operator::F64x2Fma
-            | Operator::F64x2Fms => {
+            | Operator::F64x2RelaxedMax => {


Wonder from where these guys come from.

There seems to be some new Wasm extension here:

// Relaxed SIMD operators // https://github.com/WebAssembly/relaxed-simd

maurolacy · 2023-05-24T08:43:39Z

packages/vm/src/wasm_backend/gatekeeper.rs

+            // TODO: do
+            Operator::I8x16AvgrU => todo!(),
+            Operator::I16x8AvgrU => todo!(),
+            Operator::F32x4RelaxedFma => todo!(),
+            Operator::F32x4RelaxedFnma => todo!(),
+            Operator::F64x2RelaxedFma => todo!(),
+            Operator::F64x2RelaxedFnma => todo!(),
+            Operator::I8x16RelaxedLaneselect => todo!(),
+            Operator::I16x8RelaxedLaneselect => todo!(),
+            Operator::I32x4RelaxedLaneselect => todo!(),
+            Operator::I64x2RelaxedLaneselect => todo!(),
+            Operator::I16x8RelaxedQ15mulrS => todo!(),
+            Operator::I16x8DotI8x16I7x16S => todo!(),
+            Operator::I32x4DotI8x16I7x16AddS => todo!(),
+            Operator::F32x4RelaxedDotBf16x8AddF32x4 => todo!(),


Guess this will have to be revisited soon?

Thanks for pointing this out. I sorted them now.

packages/vm/src/wasm_backend/store.rs

maurolacy

LGTM.

There are significant changes in the way memory and store are being handled in w3, with the FunctionEnv / FunctionEnvMut structs, which are above my head at this point.

maurolacy · 2023-05-24T10:01:42Z

packages/vm/src/imports.rs


        let non_existent_id = 42u32;
-        let result = do_db_next(&env, non_existent_id);
+        let result = do_db_next(fe_mut.as_mut(), non_existent_id);


Like with clone(), seems some of these as_mut() can be removed. Not exactly sure which ones.

yeah, you are right. They create short living semi-clones.

chipshort

LGTM

chipshort · 2023-05-24T09:40:01Z

packages/vm/src/cache.rs

-        }
+        // We don't load from the memory cache because we had to create new store here and
+        // serialize/deserialize the artifact to get a full clone. Could be done but adds some code
+        // for a no-so-relevant use case.


Suggested change

// for a no-so-relevant use case.

// for a not-so-relevant use case.

chipshort · 2023-05-24T09:44:54Z

packages/vm/src/cache.rs

+        // We don't load from the memory cache because we had to create new store here and
+        // serialize/deserialize the artifact to get a full clone. Could be done but adds some code
+        // for a no-so-relevant use case.


Since we don't load from memory cache anymore, the function's doc comment needs to be adjusted

chipshort · 2023-05-24T10:13:28Z

packages/vm/src/cache.rs

@@ -721,7 +730,7 @@ mod tests {
        let backend5 = mock_backend(&[]);

        // from file system
-        let _instance1 = cache
+        let _instance1: Instance<MockApi, MockStorage, MockQuerier> = cache


nit: I think this type can still be inferred.

Suggested change

let _instance1: Instance<MockApi, MockStorage, MockQuerier> = cache

let _instance1 = cache

packages/vm/src/environment.rs

chipshort · 2023-05-24T12:14:41Z

packages/vm/src/memory.rs

-            "Could not dereference this pointer to a Region",
-        )),
-    }
+    // TODO: double check error handling


Are these TODOs still relevant? The error handling looks to me, like it catches the same cases as before.

Thanks, I'll double check. I did not have the headspace to think about the error cases when trying to make everything compile.

chipshort · 2023-05-24T12:27:28Z

packages/vm/src/modules/sized_module.rs

Do we also rename the file?

chipshort · 2023-05-24T12:36:46Z

packages/vm/src/wasm_backend/store.rs

+    #[cfg(feature = "cranelift")]
+    {
+        let mut compiler = Cranelift::default();
+        for middleware in middlewares {
+            compiler.push_middleware(middleware.clone());
+        }
+        compiler.push_middleware(deterministic);
+        compiler.push_middleware(metering);
+        compiler.into()
+    }
+
+    #[cfg(not(feature = "cranelift"))]
+    {
+        let mut compiler = Singlepass::default();
+        for middleware in middlewares {
+            compiler.push_middleware(middleware.clone());
+        }
+        compiler.push_middleware(deterministic);
+        compiler.push_middleware(metering);
+        compiler.into()
    }


nit: Not really related to this PR, but this could be shortened like this:

Suggested change

#[cfg(feature = "cranelift")]

{

let mut compiler = Cranelift::default();

for middleware in middlewares {

compiler.push_middleware(middleware.clone());

}

compiler.push_middleware(deterministic);

compiler.push_middleware(metering);

compiler.into()

}

#[cfg(not(feature = "cranelift"))]

{

let mut compiler = Singlepass::default();

for middleware in middlewares {

compiler.push_middleware(middleware.clone());

}

compiler.push_middleware(deterministic);

compiler.push_middleware(metering);

compiler.into()

}

#[cfg(feature = "cranelift")]

let mut compiler = Cranelift::default();

#[cfg(not(feature = "cranelift"))]

let mut compiler = Singlepass::default();

for middleware in middlewares {

compiler.push_middleware(middleware.clone());

}

compiler.push_middleware(deterministic);

compiler.push_middleware(metering);

compiler.into()

and same in make_compile_time_store

oh, right. This was different when the code was originally written.

CosmWasm/cosmwasm#1674

This was referenced May 2, 2023

Wasmer 3.0.0-beta PoC #1438

Closed

Wasmer 3.0.0-beta PoC2 (Simon) #1467

Closed

webmaster128 added 27 commits May 4, 2023 10:02

Install Wasmer 3.2.1

980f81e

Update LimitingTunables

b548cfe

Update store.rs

63759a4

Update calls.rs

f0969c1

Update gatekeeper.rs

d9567ef

Update static_analysis.rs

1b6a555

Work

01e512c

Make it compile

9313cae

Fix memory handling

b909c6e

Update module version tests

b01b751

Update module size tests

d2f9b33

Fix write_region

48b9409

Fix store in caching

847d5b0

Fix pin_unpin_works

f25f919

Fix cache handling

754a396

Make remaining tests pass

ef4aa09

Let get_module return CachedModule

ad619fa

Fix unused warnings

b74b223

Finalize zero_padding_bytes implementation

173ac3d

Fix some more unused warnings

5810f6a

Bump min bytes version and update contracts

016df4f

Remove loupe

6d8f88d

Improve min Rust version from 1.60.0 to 1.62.0 in CI

bea904f

Remove explicit generic arguments from write_to_contract

2d3a248

Fix clippy warnings

e65df15

Make from_pinned_cache private

6361490

Return instances in multithread test

4a0d289

webmaster128 force-pushed the w3 branch from ca94c32 to 4a0d289 Compare May 4, 2023 09:49

maurolacy and others added 10 commits May 17, 2023 13:35

Restore removed move_out test

2b2b886

Restore removed comment

fe292aa

Use make_engine helper for consistency

756d1b2

Fix naming / details

e62d244

Use headless engine for simplicity

057e16e

Simplify use declaration

751442d

Co-authored-by: Simon Warta <2603011+webmaster128@users.noreply.github.com>

Set wasmer version to fixed

6dd7170

Remove memory_limit from CachedModule

9d5f735

Merge pull request #1689 from CosmWasm/w3-new-store-2

ecfafe4

W3 new store #2

Merge branch 'main' into w3

8c1e8b4

webmaster128 marked this pull request as ready for review May 23, 2023 15:41

webmaster128 added 2 commits May 23, 2023 21:14

Benchmark Wasmer 3 branch

d53b62a

Merge branch 'main' into w3

da7f99e

webmaster128 force-pushed the w3 branch from 32db2d4 to da7f99e Compare May 23, 2023 19:15

Fix some text

a0eed15

maurolacy reviewed May 24, 2023

View reviewed changes

maurolacy approved these changes May 24, 2023

View reviewed changes

webmaster128 added 5 commits May 24, 2023 12:35

Remove unused code

c8f3e4e

Improve memory doc comment

e9cc7be

Fix Modulej comment

07fcba8

Remove unused make_compile_time_store

4b5e5e3

Organize operators

fe24771

webmaster128 merged commit e45401c into main May 24, 2023
3 checks passed

webmaster128 deleted the w3 branch May 24, 2023 12:36

webmaster128 added this to the 1.3.0 milestone May 24, 2023

webmaster128 mentioned this pull request May 24, 2023

Upgrade Wasmer to Version 3 #1469

Closed

3 tasks

chipshort reviewed May 24, 2023

View reviewed changes

webmaster128 added the wasmer3/4 Changes related to the upgrade to Wasmer 3 and 4 label Jun 19, 2023

da1suk8 added a commit to da1suk8/cosmwasm that referenced this pull request Nov 8, 2023

fix: correct because the function was changed

e43cc7a

CosmWasm/cosmwasm#1674

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wasmer 3 🔥 #1674

Wasmer 3 🔥 #1674

webmaster128 commented Apr 27, 2023

webmaster128 commented May 23, 2023 •

edited

Loading

maurolacy left a comment

maurolacy May 24, 2023 •

edited

Loading

maurolacy May 24, 2023

webmaster128 May 24, 2023

maurolacy May 24, 2023

maurolacy May 24, 2023

webmaster128 May 24, 2023

maurolacy May 24, 2023

webmaster128 May 24, 2023

maurolacy left a comment •

edited

Loading

maurolacy May 24, 2023

webmaster128 May 24, 2023

chipshort left a comment

chipshort May 24, 2023

chipshort May 24, 2023

chipshort May 24, 2023

chipshort May 24, 2023

webmaster128 May 24, 2023

chipshort May 24, 2023

chipshort May 24, 2023

webmaster128 May 24, 2023

	// for a no-so-relevant use case.
	// for a not-so-relevant use case.

	let _instance1: Instance<MockApi, MockStorage, MockQuerier> = cache
	let _instance1 = cache

Wasmer 3 🔥 #1674

Wasmer 3 🔥 #1674

Conversation

webmaster128 commented Apr 27, 2023

webmaster128 commented May 23, 2023 • edited Loading

maurolacy left a comment

Choose a reason for hiding this comment

maurolacy May 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maurolacy left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chipshort left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webmaster128 commented May 23, 2023 •

edited

Loading

maurolacy May 24, 2023 •

edited

Loading

maurolacy left a comment •

edited

Loading