Skip to content
Branch: master
Find file History
ehennenfent and haxmeadroom WASM QoL Improvements (#1578)
* Add some loads and stores

* Add global operations

* Move float methods further down

* Support internal function calls

```
18:20:42 (mc) ehennenfent@nessie:~/wasm ➜ cat loop_check.c
#include <stdio.h>
#include <stdbool.h>

bool check(int arg){
  if (((arg << 2) ^ 16) == 36) {
    return true;
  }
  else{
    return false;
  }
}

int main(){
    printf("Hello world!\n");

    int g = getchar();

    printf("Got: %d\n", g);

    int y = 0;
    for (int i = 0; i < 20; i++){
      y++;

      if (y > 18){
        return check(g) ? 0 : -1;
      }
    }

    return -1;
}
18:20:54 (mc) ehennenfent@nessie:~/wasm ➜ manticore loop_check.wasm
65: i32.const (16)
11: end
65: i32.const (32)
11: end
Initialization Complete
65: i32.const (0)
65: i32.const (0)
40: i32.load (Offset 4)
65: i32.const (16)
107: i32.sub
34: tee_local (Local 1)
54: i32.store (Offset 4)
65: i32.const (32)
16: call (Func Idx 2)
Called stub function with args: (32,)
HostFunc returned [13]
26: drop
32: get_local (Local 1)
16: call (Func Idx 0)
Called stub function with args: ()
HostFunc returned [13]
34: tee_local (Local 0)
54: i32.store (Offset 0)
65: i32.const (16)
32: get_local (Local 1)
16: call (Func Idx 1)
Called stub function with args: (16, 16)
HostFunc returned [13]
26: drop
65: i32.const (0)
32: get_local (Local 1)
65: i32.const (16)
106: i32.add
54: i32.store (Offset 4)
65: i32.const (-1)
65: i32.const (0)
32: get_local (Local 0)
65: i32.const (1073741823)
113: i32.and
65: i32.const (13)
71: i32.ne
27: select
11: end
[0]
2019-07-25 18:20:58,138: [13842] m.c.manticore:INFO: Generated testcase No. 0 - test
2019-07-25 18:20:58,143: [13842] m.c.manticore:INFO: Results in /home/ehennenfent/wasm/mcore_1evptpip
```

* Improve pickle-friendliness

* Convert wasm module instructions into internal type s

* Fix class naming conflict

Thanks Pickle

* Support bitvecs as an integer type

* Refactor execution to handle single instructions at a time

* Add Atomic Stacks

It's a very inefficient implementation, but we'll make it better

* Handle Concretize Exceptions

Needs a lot of fine tuning but it's a start

* Propagate symbols all the way to return

* Add test generation infrastructure

Still very basic but it's a start

* Attempt to make Travis happy

* Fix missing generation step

* Ignore stack exhaustsion and invalid modules

* Add basic shift instructions

* Add lots of arithmetic instructions

* Fix global initialization

* Add optimization for fixed size left shift

* Make Travis tests less verbose

* Blacken

* Properly handle modulo in shifting

* Add several control flow instructions

* Blacken

* Partially Fix return instruction

* Fix missing return value from look_forward

* Update look_forward to handle nested blocks

* Fix type sigs and executor copy/paste errors, deduplicate test names

* Fix an instruction exit issue

* Fix local ordering

* Blacken

* test_address now passes for wasm
Fixed issue with json2mc.py issue with testname uniqueness
Started some work on floating point

* i hate git

* i hate git

* Adding linenumber to testcase function name
minor float progress

* f32_cmp passed for wasm

* f64_cmp passes for wasm

* f32_bitwise passes except for the problem with assertEqual not working with nan

* f64_bitwise passes except for the problem with assertEqual not working with nan

* Fixed the tests to handle [nan] comparisons

* better handling of nans and infinites

* added integer backing value for f64

* Blacken

* Blacken

* f64.nearest

* Fix argument ordering and implement N-bit memory stores

* Use subtests

* Switch print to logger

* Modify control flow instructions to match spec

"Continuation" != the thing that goes inside the Label

* Teporarily disable call instructions to prevent infinite loop

Still need to figure out control flow for nested function calls

* Blacken

* Fixed infinite loop in _call via block depth tracking

The behavior of `end` needs to depend on how many labels and activations are on the stack. Every time we push or pop a label, we increment or decrement the counter for the current call frame.

* Blacken

* Initial `br` implementation

Probably still bugged

* Support if blocks without an else instruction

* Add call_indirect

* Add grow_memory

* i32_wrap_i64

* Add several arithmetic operators

* Copy i32.ctz impl from x86 TZCNT

* Fix off-by-one in br

I think this is the correct fix (it makes all the _br tests pass) but I'm leaving the TODO there just in case.

* Blacken...

* Fix lingering issue with if block splitting

* Fix bad args to sextend in i64_extend_s_i32

* Fix indirect call null check and Float binop return types

* Improve br_table indexing

See comment

* Remove vestigial argument popping in invoke

* Fix looking forward in _return and _block

* Blacken

* Generate trap and action tests

* Fix extraneous call to exec_for_test

* Fix missing traps in load instructions

* Fix c1=0 case in ctz/clz

Hopefully this also fixes the fact that the original CTZ implementations seems to have been intended for little-endian architectures

* Handle traps in-stride during test execution

* Treat zero division as trap

* Use ctypes to convert integers

* Blacken

* Apparently I was wrong about the CTZ endianness thing

* Better range checks for offsets

This really hammers home why checking whether ea + size is _greater than_ len(mem.data) is strange if mem.data is 0-indexed, but idk, that's what the spec says.

* Make stub functions return the correct number of results

* ctypes can't handle floats

* Add option to run the start function

* Correctly handle functions with unacceptable names

* Blacken

* Treat NotImplemented as a Skip

Also counts individual subtests directly instead of screwing around with trying to grep for them in the log.

* Fix unhandled negative indices

* Fixed some integer instruction bugs

i64/32:
div_s/u
shr_s/u
shl
clz

* Fix global and memory imports

* CC

* Make default loglevel debug

* Rework conversion from int

* Make float_store work

* _actually_ fix float_store

* Restore old float creation method

* Blacken

* Add range checks to float load/store

* Force cast to unsigned in _u instructions

* Fix yet another I32 -> I64 copy paste error

How many times do we have to teach you this lesson, ~old~ young man?

* Skip problematic float tests for now

* Handle floating point inaccuracies when dividing large ints

* Skip conversion and endianness tests

* Rotl, rotl, popcnt

* Blacken

* Add some conversion stubs

* Maybe fix conversions

* Improve reinterpret and float demotion

* Re-enable endianness tests

* Include WASM logging in verbosity controls

* Propagate constraints to executor

* Blacken

* Add prototype symbolic tests

* Add api for invocation to Manticore object

* Temporarily disable native/evm tests

There's no reason to tie up Travis for 40 minutes every time I push a commit.

* Fix wasmworld import

* Concretize br_if

* Pass arg generator to invoke

* Download updated WABT

It's fixed now!

* Generate Symbolic tests (partial)

* Follow m.run format

* Blacken

* Reinitialize Manticore every 50 tests

This should cut down on the performance issues we've been seeing

* Revert "Temporarily disable native/evm tests"

This reverts commit 7abd6c1.

* I32 and I64 pass Symbolic

* Create new MC on reinit

* Blacken

* Fix return handling in test generation

* Concretize operand in `if_`

* Concretize memory addresses before access

In the long run, we should have a symbolic memory model

* Better memory access concretization (and br_table)

* Concretize float converions and indirect call operands

* Run start method

* Handle size errors in wrap+i64 and select

* Add timeout (and skip extra-problematic tests)

* Blacken

* Aggressively concretize floats

* Recast ints that should have been floats

* Better trap handling

* Unstash float concretization

* Fix log-lived 'trap' attr

* Blacken

* Concretize F32 when promoting to F64

_shouldn't_ strictly be necesssary, but the lazy conversion means that one of the tests fails. Until we get symbolic floating point, this will have to do.

* Docstrings for platforms/wasm.py

* Docstrings for ManticoreWASM

* Add docs for the module structure

* Add WASM to RTD

* missing docstring for Module itself

* Fix sphinx errors

* Bump RTD python version

I'm sorry and I swear I'll fix it later

* Reformat docstrings for sphinx

* Pacify RTD by adding newlines before params

* Missed a few

* Document the types

* Document executor

* A few more type docstrings

* Partial runtime_structure docs

* Finish documenting runtime structure

* Fix trailing whitespace

* Add collatz conjecture example

* Include undocumented members in RTD

* Add a bunch of events

* Appropriately connect event publishers

* Generate useful testcases

* Blacken

* Attempt at 3.6 backporting

* Optimize AtomicStack

* Sort solve_n output in test_solve_n

Very unclear why this is happening, but it's breaking a travis build
https://travis-ci.com/trailofbits/manticore/jobs/246451527

* Roll back to trusty?

Unsure if this is what's causing the EVM issues

* Create custom trap types

* Lazily evaluate overflows

May need to revert this if it slows down the Travis tests. It's 0.7% slower in the example I'm looking at, but that may not carry over.

* Blacken

* Address Dominik's feedback, remove stale TODOs

* Expand & Test Examples

* Fix relative path

* Fix relative path?

* capitalize Path

* Fix relative path for real

* Blacken...

* Test finalize

* Add supplemental tests for coverage

* Use dedicated helper for store access

* Handle quoted strings in imported module names

* Allow WASMWorld to hold multiple modules

Necessary for getting imports/exports to work, but still needs some changes before it passes the tests.

* Actually use export maps

* Fix most of the control flow issues with import tests

* Use available cores for test generation

* Escape module name

* Don't reinit modules that have already been initialized

* Use addresses for imports instead of copying

* Fix removed timeout argument

* Interleave registration and aliases

* Fix broken "None" check for address

* Blacken

* Make manual exports work with address syntax

* Import all registered modules

Only necessary for elem, but it's tricky to make changes just for one test, so *shrug emoji*

* Remove stringified annotations

Turns out those only work in the same file...

* Fix easy type issues in all but runtime structure

* Fix easy type issues in runtime_structure

* Fix* everything but imports

* Blacken

* Blacken 2

* Typo

* Merge module structure and runtime structure

* Fix instance vs type confusion

* Fix mypy and concrete tests

Still an issue with the symbolic tests

* Still generate symbolic tests though

I keep forgetting not to commit that file...

* Attempt to fix env for symbolic tests

* Blacken/mypy

* Fix missing supplemental environment variables arg

* Fix docstrings for new import style

* CC

* Bump timeout duration to 70 minutes

I don't like it, but adding more jobs to Travis isn't going to make our builds any faster

* Fix pycharm type, import, and docstring errors

* Explain type: ignore

* Missed two type:ignore's

* Fix CLI so it doesn't throw up on arbitrary modules

* Add getters for locals and globals to state

* Make host functions take the state instead of the constriant set

* Support __getattr__ style function calling

* Implicitly run Manticore when invoked via __getattr__

* Automatically reinitialize before __getattr__ style function calls

* Fix docstring

* Fix mypy?

* Fix test getchar impl

* Add tests for implicit call and main function

* Fix wrong memory callback

* Okay fine I'll check the output

* Add missing jinja dep

Sphinx includes it, but we should do so as well

* Bump spec tests version

* Test multiple implicit calls

* Split symbolic/concrete WASM tests

* Add some missing docstrings

* Hopefully fix wasm test split?

* Properly split symbolic tests

* Fix fallthrough

Okay I swear it's fixed this time

* Add if_check example

* Blacken

* Add WASM Example to README

Co-authored-by: HaxMeadroom <haxmeadroom@gmail.com>
Latest commit 83325f9 Jan 3, 2020
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
collatz
if_check
You can’t perform that action at this time.