Add support for latest Intel Pin (3.7)

[RKX1209]

Ported taint analysis library libdft64 for PinCRT which is used as a C/C++ runtime framework in Intel Pin (>=3.0).
There are some changes in both components libdft64 and vuzzer64.

libdft64

• Change build method to use default Makefile from Pin
• Port source codes for PinCRT.

vuzzer64

• Add bb shell script (In some distributions, python's subprocess doesn't work correctly for hard coded "--" (double dash).
• Use latest pin binary (pin.sh was deprecated)

[tosanjay]

Hi Ren,
This is really great. Just a quick query: Have you done some testing on couple of applications? If so, can you please let us know which applications you tested with?
Again, thanks for this patch :)

[RKX1209]

Hi. Thanks for comment.
Yes. Currently I've done simple tests for libdft64 with Intel Pin 3.7.
I've checked the output of run_2.sh against some binaries like comp(rebuild with 64bit env), /usr/bin/who, /usr/bin/base64 on my Ubuntu 18.04 (linux kernel 4.15).

Most simple example, a comp binary

$ ./run_2.sh "path_to_vuzzer64/samples/comp path_to_vuzzer64/samples/comp.in" comp.in 0
$ cat path_to_vuzzer64/samples/comp.in
ABCD
$ cat cmp.out
8 reg reg 0x0000000000400671 {0} {} {} {} {} {} {} {} {2} {} {} {} {} {} {} {} 0x41 0x43 
8 reg reg 0x0000000000400692 {2} {} {} {} {} {} {} {} {3} {} {} {} {} {} {} {} 0x43 0x44
$ cat ../libdft64/tools/pintool.log
...
In open
 in_dtracker_whitelist path_to_vuzzer64/samples/comp.in
 Inserted path_to_vuzzer64/samples/comp.in 3.
  CMP 0x00007fffdf02ac8e cmp rax, 0xfffffffffffff000
 Setting taint 3 4
  CMP 0x00007fffdf02b081 cmp rax, 0xfffffffffffff000
  CMP 0x0000000000400671 cmp dl, al
 IN PRINT LOG
  CMP 0x0000000000400692 cmp dl, al
 IN PRINT LOG
....
 close 3

And other binaries works correctly as a comp example.
I also ran vuzzer64 fuzzer and it seems works fine.

So currently I think vuzzer64 can work with Intel Pin3.7 and you can merge it.
If I find some failure which can happen with latest Pin, I'll send additional PR to fix it.

Thanks:)

[RKX1209]

Hi. Is there any progress?

[tosanjay]

Hi,I will merge it soon (as currently i am traveling). as of now, it seems promising.   Thanks & Regards - Sanjay https://sites.google.com/site/securitythoughts/ On Wednesday, November 28, 2018, 2:53:30 AM GMT+5:30, Ren Kimura <notifications@github.com> wrote: Hi. Is their any progress? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[r1ce-m]

While compiling libdft64 this happens. Any idea why?

In file included from tagmap.h:38:0,
from tagmap.cpp:41:
/usr/include/ewah.h: In instantiation of ‘const uword* EWAHBoolArrayRawIterator::dirtyWords() const [with uword = unsigned int]’:
/usr/include/runninglengthword.h:439:69: required from ‘uword BufferedRunningLengthWord::getLiteralWordAt(size_t) [with uword = unsigned int; size_t = long unsigned int]’
/usr/include/ewah.h:1877:52: required from ‘void EWAHBoolArray::logicalor(const EWAHBoolArray&, EWAHBoolArray&) const [with uword = unsigned int]’
tagmap.cpp:67:27: required from here
/usr/include/ewah.h:1339:22: error: ‘const class std::vector<unsigned int, std::allocator >’ has no member named ‘data’; did you mean ‘at’?
return myparent->data() +
~~~~~~~~~~^~~~
at
/home/natalie/Research/VUzzer64/pin-3.7/source/tools/Config/makefile.default.rules:196: recipe for target 'obj-intel64/tagmap.o' failed
make[1]: *** [obj-intel64/tagmap.o] Error 1
make[1]: Leaving directory '/home/natalie/Research/VUzzer64/vuzzer64_pin3/libdft64'
/home/natalie/Research/VUzzer64/pin-3.7/source/tools/Config/makefile.config:328: recipe for target 'all' failed
make: *** [all] Error 2

[RKX1209]

Sorry, I forgot to note that PinCRT doesn't support some C++11 features like vector->data().
We need to use older version of EWAHBoolArray (actually I used ver 0.4.0)

[tosanjay]

Hi,
your pull request is merged. Thank you very much :)

[vanhauser-thc]

the README still says PIN 2.13 is required - is that outdated or are there othe requirements why it still must be that old version?