How to install pyvex in Windows?

[vancaho]

Hi, I want to use pyvex in Windows. However after I install pyvex by executing "pip install pyvex", and I find that the "pyvex_static.so" cannot be loaded on windows platform because it is an ELF file while windows requires a PE file. So How to compile the module in windows? Thanks

[vancaho]

I build a pyvex_Static.so with cygwin and mingw in windows successfully. However after I install pyvex with this new library and run "import angr", the follow errors were displayed:

Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\angr__init__.py", line 7, in
from .project import *
File "C:\Python27\lib\site-packages\angr\project.py", line 10, in
import cle
File "C:\Python27\lib\site-packages\cle__init__.py", line 6, in
from .loader import *
File "C:\Python27\lib\site-packages\cle\loader.py", line 688, in
from .tls import TLSObj
File "C:\Python27\lib\site-packages\cle\tls.py", line 4, in
from .backends import Backend
File "C:\Python27\lib\site-packages\cle\backends__init__.py", line 4, in
import archinfo
File "C:\Python27\lib\site-packages\archinfo__init__.py", line 2, in

from .arch import *

File "C:\Python27\lib\site-packages\archinfo\arch.py", line 318, in
ArchAMD64(), ArchX86(),
File "C:\Python27\lib\site-packages\archinfo\arch_amd64.py", line 10, in ini
t
super(ArchAMD64, self).init(endness)
File "C:\Python27\lib\site-packages\archinfo\arch.py", line 19, in init
self.vex_archinfo = self.vex_archinfo.copy()
AttributeError: 'NoneType' object has no attribute 'copy'

what does this error implies? Thanks very much

[bannsec]

I'm actually having the same problem. I generally use it on Linux, but my desktop (non-VM) runs Windows. It appears that pyVex is designed to build either in Mac OSX or Nix. The main problem is that it ends up building the .so file which is a Linux module and would obviously not load correctly into Windows. What isn't clear to me is even if we were able to get that to build into a dll, would the rest of the code that relies on it actually load it and utilize it correctly. My guess is no.

[zardus]

I think that at some point, someone had pyvex working in Windows. I don't have a Windows dev env, though, so I can't really look into this :-(

[rhelmot]

I got pyvex running on Windows at one point! I used a mingw build
environment. It's theoretically totally possible since VEX is totally
platform agnostic, but the pyvex build process might have been altered in a
way that breaks Windows.

On Thursday, February 4, 2016, Yan notifications@github.com wrote:

I think that at some point, someone had pyvex working in Windows. I don't
have a Windows dev env, though, so I can't really look into this :-(

—
Reply to this email directly or view it on GitHub
#19 (comment).

[bannsec]

OK, think I got it to compile and everything. Problem now is I'm getting this error when I try to run it:

cffi.api.CDefError: cannot parse "extern VexControl vex_control;"

I don't have much knowledge on cffi. Thoughts?

[bannsec]

found the error. i think there's an error in the make_ffi as well since I only have 1 line (the one mentioned) in that file. Gonna look into that.

[rhelmot]

Can you make sure that your cffi is up to date? mine is version 1.4.2. We've had issues in the past with cffi version mismatches messing up the parsing.

[bannsec]

It's 1.5.0. Am I correct in understanding there should likely be more than just the extern line? I'm guessing there should at least be something defining VexControl as a class.

[rhelmot]

Yes, it's in vex/pub/libvex.h.

[zardus]

There should be a lot of stuff in there. Mine's 1358 lines long, for
example... What does make_ffi output when you run it manually?

On Fri, Feb 5, 2016 at 7:00 PM, Andrew Dutcher notifications@github.com
wrote:

Yes, it's in vex/pub/libvex.h.

—
Reply to this email directly or view it on GitHub
#19 (comment).

[bannsec]

figured it out:

header,_ = subprocess.Popen(['cpp', '-I'+vex_path, 'pyvex_c/pyvex_static.h'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT).communicate()

Windows doesn't understand forward slashes. Switched that to backslash and raw and it worked. It really should be an os.join.path to be compatible all around. I've found a bunch of other gotchas that I'll submit a pull for once I actually get this thing working. Next problem..

OSError: cannot load library pyvex..\pyvex_c\pyvex_static.dll: error 0xc1

[rhelmot]

I don't know about the OSError, but my understanding is that windows understands forward slashes just fine. The cpp implementation might not, though... thanks for figuring that out.

[bannsec]

Got it running (i think). Turns out i need to be running the 32-bit python version due to the 32-bit dll build. Couldn't get 64-bit to build for some reason. Is there a quick way to tell if pyvex is working by itself as I haven't built anything else yet?

[rhelmot]

OH hang on there was a bug we had building z3 on windows where there was a special flag that had to be passed to the build to build with the right number of bits

import pyvex, archinfo
pyvex.IRSB('\xc3', 0, archinfo.ArchAMD64()).pp()

Pyvex depends on archinfo but if you have that this should work

[rhelmot]

http://stackoverflow.com/questions/34810202/why-cant-i-install-angr-z3-using-pip-on-windows-when-python-is-64-bit

This is the relevant issue. I'm not familiar with the pyvex build process but maybe there's an analogous line that needs changing?

[bannsec]

Eh, the main problem I'm finding is that gcc for MinGW is 32-bit and won't compile 64. MinGW-64 is spitting out compile errors for some reason...

I've uploaded a (hopefully) working wheel for Windows pyvex python-32bit here: https://github.com/Owlz/angr-Windows/tree/master/pyvex/python32

[bannsec]

OK. Got everything building correctly now and will submit a pull. However, the file "vex" in the root of this repo causes problems with the setup thinking vex is already installed. If I rename that file then I've got the Windows side building. Do you know why that file is there? Is there any concern with moving it/renaming it?

[bannsec]

So this is weird... It builds fine and all now, and loads fine. But there's some non-deterministic part of it. If i use the following as my test script:

import pyvex
import archinfo

# translate an AMD64 basic block (of nops) at 0x400400 into VEX
irsb = pyvex.IRSB("\x90\x90\x90\x90\x90", 0x400400, archinfo.ArchAMD64())

# pretty-print the basic block
print(irsb.pp())

Most of the time when I run it it will not return anything (crashing). Randomly it will return the irsb.pp() correctly. I do nothing different from run to run. Also, python crashes so hard that python's pdb won't even catch the crash. I can't figure out exactly why this is happening, but I know it's happening at the following line:

c_irsb = pvc.vex_block_bytes(vex_arch, arch.vex_archinfo, c_bytes + bytes_offset, mem_addr, num_bytes, 1)

So it has the feel of ASLR or something... :-/

[rhelmot]

I'm going to guess that this has been fixed at some point with all the nonsense we've been doing to pyvex and vex!