Can't build on Mac

[hebnern]

I initially tried installing angr via pip, but was getting an error about not being able to find pyvex_static.dylib. It seems that the pip package includes the linux pyvex_static.so instead of the mac dylib. So I tried building pyvex myself, but I am getting:

Installing collected packages: pyvex
  Running setup.py develop for pyvex
    Complete output from command /usr/local/opt/python/bin/python2.7 -c "import setuptools, tokenize; __file__='/Users/hebner/git/pyvex/setup.py'; exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps:
    running develop
    Building libVEX
    EXTRA_CFLAGS="-fPIC -w" make -f Makefile-gcc
    make[1]: Nothing to be done for `all'.
    Building pyvex-static
    make: `pyvex_static.dylib' is up to date.
    Creating CFFI defs file
    No handlers could be found for logger "cffier"
    ./vex-dev/pub
    Traceback (most recent call last):
      File "make_ffi.py", line 81, in <module>
        doit(sys.argv[1])
      File "make_ffi.py", line 71, in doit
        new_good = find_good_scan(good, remaining[1:])
      File "make_ffi.py", line 53, in find_good_scan
        return find_good_scan(known_good, questionable[:fail-2-len(known_good)])
      File "make_ffi.py", line 46, in find_good_scan
        raise Exception("Unknown error")
    Exception: Unknown error
    error: Unable to generate cffi file.

The specific exception that is generated is: <class 'cffi.api.CDefError'> setjmp arg 1: unknown type 'jmp_buf' (if you meant to use the old C syntax of giving untyped arguments, it is not supported)

I'm not really sure what make_ffi.py is doing, so wasn't sure how to fix this error.

[zardus]

So, make_ffi.py is a gigantic hack that coerces the VEX headers to work with CFFI. The most common reason for this failing is that your pycparser or cffi is not up-to-date. Can you post the output of pip freeze? Alternatively, run pip install -U cffi pycparser in your virtualenv and try again with the updated versions.

The wheel does, indeed, ship for Linux. There might be a way to tell other platforms not to use it; I gotta look into that.

[hebnern]

Thanks for the info. I checked my dependency versions after reading through #14, but they are all up to date.

I did a bit more digging and the root of the issue seems to be that jmp_buf is defined as an array of ints with a size that consists of an expression instead of a constant on my system, which pycparser doesn't support. make_ffi.py removes this line, and then subsequent references to the jmp_buf type cannot be parsed. I tried excluding all lines that contain jmp_buf, and was able to build. However when I import angr I get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/angr/__init__.py", line 7, in <module>
    from .project import *
  File "/usr/local/lib/python2.7/site-packages/angr/project.py", line 10, in <module>
    import cle
  File "/usr/local/lib/python2.7/site-packages/cle/__init__.py", line 6, in <module>
    from .loader import *
  File "/usr/local/lib/python2.7/site-packages/cle/loader.py", line 710, in <module>
    from .tls import TLSObj
  File "/usr/local/lib/python2.7/site-packages/cle/tls.py", line 4, in <module>
    from .backends import Backend
  File "/usr/local/lib/python2.7/site-packages/cle/backends/__init__.py", line 4, in <module>
    import archinfo
  File "/usr/local/lib/python2.7/site-packages/archinfo/__init__.py", line 2, in <module>
    from .arch import *
  File "/usr/local/lib/python2.7/site-packages/archinfo/arch.py", line 7, in <module>
    import pyvex as _pyvex
  File "/Users/hebner/git/pyvex/pyvex/__init__.py", line 72, in <module>
    _op_types = { _:_get_op_type(_) for _ in enums_to_ints if _.startswith('Iop_') and _ != 'Iop_INVALID' and _ != 'Iop_LAST' }
  File "/Users/hebner/git/pyvex/pyvex/__init__.py", line 72, in <dictcomp>
    _op_types = { _:_get_op_type(_) for _ in enums_to_ints if _.startswith('Iop_') and _ != 'Iop_INVALID' and _ != 'Iop_LAST' }
  File "/Users/hebner/git/pyvex/pyvex/__init__.py", line 68, in _get_op_type
    irsb = pvc.emptyIRSB()
  File "/usr/local/lib/python2.7/site-packages/cffi/api.py", line 690, in __getattr__
    make_accessor(name)
  File "/usr/local/lib/python2.7/site-packages/cffi/api.py", line 686, in make_accessor
    make_accessor_locked(name)
  File "/usr/local/lib/python2.7/site-packages/cffi/api.py", line 680, in make_accessor_locked
    raise AttributeError(name)
AttributeError: emptyIRSB

A bit more digging revealed that IRSB related types were being removed due to the {} NoOp; replacement failing due to different indentation on my system. I replaced the simple string.replace() with a re.sub() to account for different amounts of whitespace, and then things started working. While I was at it, I found that the attempted removal of lines that start with # was failing due to indentation of some preprocessor directives. I changed this check to '#' in line as a workaround. I'll submit a patch with these changes.

[hebnern]

Ran into a few more issues: cpp was converting const to __const, so I am converting it back afterwards, and some __attribute__ ((noreturn))'s were causing issues, so they are being removed as well.

However when I try to do a CFG analysis, I am now getting:

Traceback (most recent call last):
  File "cg.py", line 12, in <module>
    cfg = b.analyses.CFG(start=0x10ecd, keep_state=True, context_sensitivity_level=3)
  File "/usr/local/lib/python2.7/site-packages/angr/analysis.py", line 84, in make_analysis
    oself.__init__(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/angr/analyses/cfg.py", line 245, in __init__
    self._construct()
  File "/usr/local/lib/python2.7/site-packages/angr/analyses/cfg.py", line 415, in _construct
    non_returning_functions)
  File "/usr/local/lib/python2.7/site-packages/angr/analyses/cfg.py", line 1124, in _handle_entry
    current_function_addr=current_function_addr)
  File "/usr/local/lib/python2.7/site-packages/angr/analyses/cfg.py", line 874, in _get_simrun
    sim_run = self.project.factory.sim_run(current_entry.state, jumpkind=jumpkind)
  File "/usr/local/lib/python2.7/site-packages/angr/factory.py", line 120, in sim_run
    r = self.sim_block(state, addr=addr, **block_opts)
  File "/usr/local/lib/python2.7/site-packages/angr/factory.py", line 59, in sim_block
    **block_opts)
  File "/usr/local/lib/python2.7/site-packages/angr/lifter.py", line 116, in lift
    traceflags=traceflags)
  File "/Users/hebner/git/pyvex/pyvex/__init__.py", line 136, in __init__
    c_irsb = pvc.vex_block_bytes(vex_arch, arch.vex_archinfo, c_bytes + bytes_offset, mem_addr, num_bytes, 1)
KeyError: 'x86_cr0'

Thoughts?

[hebnern]

Looks like there is already an angr issue open for this one: angr/angr#46

[ronnychevalier]

And this issue points here: #20

can you try to update archinfo and pyvex?

[zardus]

So @salls managed to track the x86_cr0 error down! Can you merge with the latest pyvex and try again?

[hebnern]

That seems to have fixed it. Thanks!

[zardus]

w00t w00t