New off-chain engine: Implement cross-contract methods

[cmichi]

Needed for #565.

• call
• instantiate

[KaiserKarel]

Is mentoring offered on this issue? It seems quite difficult, but I'd be willing to spend some time on it.

[Robbepop]

Indeed implementing this will be very difficult. I personally am out of ideas how to properly implement this given our current off-chain engine architectures. @cmichi do you have ideas?

@KaiserKarel What is your motivation to have this? Do you have some ideas in mind for the implementation?

[JayPavlina]

I need this to run tests that require cross contract calls. Unfortunately, most of my contracts use other contracts, usually ERC20 or ERC721, so I have to test with redspot, which I find to be a huge drain on productivity. Being able to write all tests in rusts would be amazing. If this issue cannot be solved, I would love to know if there is some kind of workaround to be able to test contracts that call other contracts in rust.

[Robbepop]

[...] Unfortunately, most of my contracts use other contracts [...]

Yep, this is kind of expected and normal with more complex smart contracts. The off-chain environments that have been implemented so far for ink! are limited to some extend. Also ink! generates very different code for both Wasm and off-chain compilation which was required to make limited off-chain testing simpler but which naturally makes it more difficult to emulate on-chain properly in an off-chain environment.
Given our limitations we already came quite a long way with the most recent off-chain environment implementation.

I don't think it is impossible to implement proper off-chain testing with cross-contract call support but it might eventually require a different architecture for our off-chain environments - not sure about this though.

The problem is that if you compile your smart contract for off-chain testing you do not compile it as a WebAssembly blob but instead as a standard Rust (x86 or so) library. If the contract is compiled with ink-as-dependency there no longer is a big difference when compiling for off-chain testing and we are able to directly include a depended-on contract into the root contract, so we actually are not using any external environment to do calls but use very efficient standard calling procedures. It is literally just incorporated into the root contract directly with all of its state instead of being put there as a direction via type-safe AccountId as it was the case as Wasm compilation.

[JayPavlina]

I had some success before when I compiled the dependency contract without the ink-as-dependency feature when I was testing, but I never found a setup that also allowed me to also compile with cargo contract. Since features are additive, it's difficult to remove the ink-as-dependency feature only during testing.

[h4x3rotab]

Is it a work in progress or still in waiting?

Recently we are testing a project with multiple contracts. So far the most awkward situations are (1) no way to make cross-contract calls, and (2) contract address is not handled. This makes testing multi contract projects very hard. Since a rewrite of the off-chain engine may take a long time, now we are trying to mitigate by conditional compilation.

The idea is simple. We can just mock the contract reference by a contract object instantiated in a unit test. Then at each place where an inter-contract invocation happens, we use the conditional compilation to execute the direct call on the contract object. Here's the sample code:

// ...

impl Contract {
  #[ink(message)]
  fn do_something(&self) {
    // ...
    // Here is the trick: we call the mock object when in test
    #[cfg(not(test))]
    let r = self.remote.transfer(b)
    #[cfg(test)]
    let r = tests::mock_remote_contract::with(|remote| remote.transfer(b)).unwrap();
  }
}

#[cfg(test)]
mod tests {
  // Define an `environmental` variable to easily pass a mock object to the contract.
  environmental!(mock_remote_contract: RemoteContract);
  #[test]
  fn cross_contract_invocation_works() {
    let mut remote = RemoteContract::new();
    let mut contract = Contract::new();

    // When a cross-contract invocation happens, we wrap it like this:
    mock_remote_contract::using(&mut remote, || {
      contract.do_something()
    });
  }
}

I haven't digged too deep to this. But without any large scale refactor, it looks like we can also do a similar trick to better simulate the inter contract calls. For example, we can write a macro like this to wrap the invocation expression: remote_call!(contract_ref.method(arg0, arg1, ...))

Then in the test environment, we can define a virtual call stake. Whenever a contract call is made, we push a new environment to the stack with the contract address. And since now we have a stack, we can correctly tract the caller and callee (i.e. the target contract address). Finally, when the call returns, we pop the environment from the stack.

Of course a better way is to maintain the environment stack by the code generated by #[ink(message)] for the unit test target.

[h4x3rotab]

I've implemented a PoV of macro based cross-contract mock demo in OpenBrush: Supercolony-net/openbrush-contracts#136

[HCastano]

This was made irrelevant by the E2E testing framework introduced in ink! 4.0.